Duckietown Challenges Home Challenges Submissions

Evaluator 4893

ID4893
evaluatornogpu-prod-03
ownerI don't have one πŸ˜€
machinenogpu-prod_0566ecaa07db
processnogpu-prod-03_0566ecaa07db
version6.2.5
first heard
last heard
statusinactive
# evaluating
# success15 53957
# timeout1 57520
# failed4 54661
# error
# aborted2 57205
# host-error43 53940
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
5752010903Ayman ShamsΒ πŸ‡¨πŸ‡¦exercise_ros_templateaido5-LF-sim-validationLFv-simtimeoutyesnogpu-prod-03----No reset possible
5745810918Dishank BansalΒ πŸ‡¨πŸ‡¦sim-exercise-1aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-030:17:22
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.9580520669099823
survival_time_median16.5000000000001
deviation-center-line_median0.33632965881590027
in-drivable-lane_median11.100000000000104


other stats
agent_compute-ego0_max0.013349967532687718
agent_compute-ego0_mean0.013019779026474653
agent_compute-ego0_median0.013077475734815526
agent_compute-ego0_min0.01257419710357984
complete-iteration_max0.20687710338168672
complete-iteration_mean0.18507349846316223
complete-iteration_median0.1862126183354758
complete-iteration_min0.16099165380001068
deviation-center-line_max0.5581988284647905
deviation-center-line_mean0.36246439920842505
deviation-center-line_min0.2189994507371091
deviation-heading_max2.0851816254028286
deviation-heading_mean1.629678609599893
deviation-heading_median1.779299425032109
deviation-heading_min0.8749339629325252
driven_any_max9.43240075717308
driven_any_mean4.033454871695407
driven_any_median2.5891698063729978
driven_any_min1.5230791168625557
driven_lanedir_consec_max2.04677513629244
driven_lanedir_consec_mean1.173152232057151
driven_lanedir_consec_min0.7297296581161998
driven_lanedir_max2.04677513629244
driven_lanedir_mean1.173152232057151
driven_lanedir_median0.9580520669099823
driven_lanedir_min0.7297296581161998
get_duckie_state_max1.3696650664011638e-06
get_duckie_state_mean1.349980204508626e-06
get_duckie_state_median1.356297868899352e-06
get_duckie_state_min1.3176600138346354e-06
get_robot_state_max0.0037495291457985934
get_robot_state_mean0.0036514645375311625
get_robot_state_median0.003634119265787431
get_robot_state_min0.0035880904727511936
get_state_dump_max0.004698077131365927
get_state_dump_mean0.004631867777626119
get_state_dump_median0.00464789757938505
get_state_dump_min0.004533598820368449
get_ui_image_max0.036290452745225694
get_ui_image_mean0.031245016495824725
get_ui_image_median0.030944901404329542
get_ui_image_min0.026799810429414112
in-drivable-lane_max42.49999999999908
in-drivable-lane_mean16.94999999999983
in-drivable-lane_min3.1000000000000334
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 9.43240075717308, "get_ui_image": 0.028885832671476815, "step_physics": 0.10151848534315518, "survival_time": 53.3999999999991, "driven_lanedir": 2.04677513629244, "get_state_dump": 0.004698077131365927, "get_robot_state": 0.00367847305240756, "sim_render-ego0": 0.0039452427006749155, "get_duckie_state": 1.3484366035996697e-06, "in-drivable-lane": 42.49999999999908, "deviation-heading": 1.97112371815148, "agent_compute-ego0": 0.0130488707486216, "complete-iteration": 0.1690045605205397, "set_robot_commands": 0.0021863014927748993, "deviation-center-line": 0.5581988284647905, "driven_lanedir_consec": 2.04677513629244, "sim_compute_sim_state": 0.008988964546496229, "sim_compute_performance-ego0": 0.001967008142854924}, "LF-norm-zigzag-000-ego0": {"driven_any": 3.5547943540204026, "get_ui_image": 0.036290452745225694, "step_physics": 0.13015321254730225, "survival_time": 22.450000000000184, "driven_lanedir": 0.7645279993932639, "get_state_dump": 0.004619151751200358, "get_robot_state": 0.0035880904727511936, "sim_render-ego0": 0.0039983214272393125, "get_duckie_state": 1.3176600138346354e-06, "in-drivable-lane": 17.050000000000196, "deviation-heading": 1.5874751319127376, "agent_compute-ego0": 0.013349967532687718, "complete-iteration": 0.20687710338168672, "set_robot_commands": 0.002159208721584744, "deviation-center-line": 0.26075669759770465, "driven_lanedir_consec": 0.7645279993932639, "sim_compute_sim_state": 0.010613026089138456, "sim_compute_performance-ego0": 0.002019890149434407}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.6235452587255932, "get_ui_image": 0.03300397013718227, "step_physics": 0.1316724678255477, "survival_time": 10.550000000000017, "driven_lanedir": 1.1515761344267008, "get_state_dump": 0.004676643407569741, "get_robot_state": 0.0037495291457985934, "sim_render-ego0": 0.00391862189994668, "get_duckie_state": 1.3641591341990343e-06, "in-drivable-lane": 3.1000000000000334, "deviation-heading": 2.0851816254028286, "agent_compute-ego0": 0.013106080721009453, "complete-iteration": 0.20342067615041193, "set_robot_commands": 0.0022298423749095987, "deviation-center-line": 0.41190262003409583, "driven_lanedir_consec": 1.1515761344267008, "sim_compute_sim_state": 0.00902235170580306, "sim_compute_performance-ego0": 0.0019558420721090064}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.5230791168625557, "get_ui_image": 0.026799810429414112, "step_physics": 0.1000998926659425, "survival_time": 9.55, "driven_lanedir": 0.7297296581161998, "get_state_dump": 0.004533598820368449, "get_robot_state": 0.0035897654791673026, "sim_render-ego0": 0.00394614040851593, "get_duckie_state": 1.3696650664011638e-06, "in-drivable-lane": 5.150000000000008, "deviation-heading": 0.8749339629325252, "agent_compute-ego0": 0.01257419710357984, "complete-iteration": 0.16099165380001068, "set_robot_commands": 0.002204009642203649, "deviation-center-line": 0.2189994507371091, "driven_lanedir_consec": 0.7297296581161998, "sim_compute_sim_state": 0.005180823306242625, "sim_compute_performance-ego0": 0.0019784035782019296}}
set_robot_commands_max0.0022298423749095987
set_robot_commands_mean0.002194840557868223
set_robot_commands_median0.002195155567489274
set_robot_commands_min0.002159208721584744
sim_compute_performance-ego0_max0.002019890149434407
sim_compute_performance-ego0_mean0.001980285985650067
sim_compute_performance-ego0_median0.001972705860528427
sim_compute_performance-ego0_min0.0019558420721090064
sim_compute_sim_state_max0.010613026089138456
sim_compute_sim_state_mean0.008451291411920093
sim_compute_sim_state_median0.009005658126149646
sim_compute_sim_state_min0.005180823306242625
sim_render-ego0_max0.0039983214272393125
sim_render-ego0_mean0.00395208160909421
sim_render-ego0_median0.003945691554595423
sim_render-ego0_min0.00391862189994668
simulation-passed1
step_physics_max0.1316724678255477
step_physics_mean0.11586101459548692
step_physics_median0.11583584894522872
step_physics_min0.1000998926659425
survival_time_max53.3999999999991
survival_time_mean23.987499999999827
survival_time_min9.55
No reset possible
5735210952Fernanda Custodio Pereira do CarmoΒ πŸ‡¨πŸ‡¦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-030:17:38
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median2.520121143073136
survival_time_median25.95000000000011
deviation-center-line_median1.1625065828289225
in-drivable-lane_median8.074999999999761


other stats
agent_compute-ego0_max0.013855936350645843
agent_compute-ego0_mean0.01313804203593464
agent_compute-ego0_median0.013056341977063922
agent_compute-ego0_min0.01258354783896488
complete-iteration_max0.21908065124794288
complete-iteration_mean0.17991153786115077
complete-iteration_median0.16907461886472056
complete-iteration_min0.1624162624672191
deviation-center-line_max3.917121392328863
deviation-center-line_mean1.5960210501198118
deviation-center-line_min0.14194964249253816
deviation-heading_max6.302288303368849
deviation-heading_mean3.456316185810077
deviation-heading_median3.284581572438045
deviation-heading_min0.953813294995368
driven_any_max9.344885231590265
driven_any_mean4.459112795092815
driven_any_median3.908455255797831
driven_any_min0.674655437185334
driven_lanedir_consec_max7.167994682911328
driven_lanedir_consec_mean3.113692901215736
driven_lanedir_consec_min0.24653463580534396
driven_lanedir_max7.167994682911328
driven_lanedir_mean3.113692901215736
driven_lanedir_median2.520121143073136
driven_lanedir_min0.24653463580534396
get_duckie_state_max1.5992506854639365e-06
get_duckie_state_mean1.4452994996583046e-06
get_duckie_state_median1.4751428260360902e-06
get_duckie_state_min1.2316616610971013e-06
get_robot_state_max0.0039766982749656395
get_robot_state_mean0.00385333321618148
get_robot_state_median0.003903419406224403
get_robot_state_min0.003629795777311473
get_state_dump_max0.004815833157643391
get_state_dump_mean0.004711983339446032
get_state_dump_median0.00472050655952354
get_state_dump_min0.004591087081093661
get_ui_image_max0.03527246801941483
get_ui_image_mean0.030018087989622723
get_ui_image_median0.02911190400797046
get_ui_image_min0.02657607592313514
in-drivable-lane_max14.899999999999965
in-drivable-lane_mean8.37499999999988
in-drivable-lane_min2.450000000000035
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 2.4108353868072405, "get_ui_image": 0.027264448647078773, "step_physics": 0.095488635964669, "survival_time": 16.400000000000098, "driven_lanedir": 2.104852527822607, "get_state_dump": 0.004643238786506073, "get_robot_state": 0.0038549349300767153, "sim_render-ego0": 0.00399887815434882, "get_duckie_state": 1.4160179439649031e-06, "in-drivable-lane": 2.450000000000035, "deviation-heading": 1.7708609002199542, "agent_compute-ego0": 0.012925777754160407, "complete-iteration": 0.1624162624672191, "set_robot_commands": 0.002298529749583328, "deviation-center-line": 0.7429742981677188, "driven_lanedir_consec": 2.104852527822607, "sim_compute_sim_state": 0.009733290657808716, "sim_compute_performance-ego0": 0.0021090804624702432}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.674655437185334, "get_ui_image": 0.03527246801941483, "step_physics": 0.141489428502542, "survival_time": 5.349999999999989, "driven_lanedir": 0.24653463580534396, "get_state_dump": 0.004797774332541006, "get_robot_state": 0.0039766982749656395, "sim_render-ego0": 0.0040233201450771755, "get_duckie_state": 1.5342677081072772e-06, "in-drivable-lane": 3.2499999999999885, "deviation-heading": 0.953813294995368, "agent_compute-ego0": 0.013855936350645843, "complete-iteration": 0.21908065124794288, "set_robot_commands": 0.002355553485729076, "deviation-center-line": 0.14194964249253816, "driven_lanedir_consec": 0.24653463580534396, "sim_compute_sim_state": 0.011082260696976274, "sim_compute_performance-ego0": 0.0021213359302944606}, "LF-norm-techtrack-000-ego0": {"driven_any": 5.406075124788422, "get_ui_image": 0.030959359368862148, "step_physics": 0.10297528332631128, "survival_time": 35.50000000000012, "driven_lanedir": 2.9353897583236654, "get_state_dump": 0.004591087081093661, "get_robot_state": 0.003629795777311473, "sim_render-ego0": 0.003829383313572021, "get_duckie_state": 1.2316616610971013e-06, "in-drivable-lane": 14.899999999999965, "deviation-heading": 4.798302244656136, "agent_compute-ego0": 0.01258354783896488, "complete-iteration": 0.17486361444583254, "set_robot_commands": 0.00217073275570125, "deviation-center-line": 1.582038867490126, "driven_lanedir_consec": 2.9353897583236654, "sim_compute_sim_state": 0.012064422736188027, "sim_compute_performance-ego0": 0.0019686098004862897}, "LF-norm-small_loop-000-ego0": {"driven_any": 9.344885231590265, "get_ui_image": 0.02657607592313514, "step_physics": 0.09945138288874313, "survival_time": 59.99999999999873, "driven_lanedir": 7.167994682911328, "get_state_dump": 0.004815833157643391, "get_robot_state": 0.003951903882372092, "sim_render-ego0": 0.004113445869591909, "get_duckie_state": 1.5992506854639365e-06, "in-drivable-lane": 12.899999999999531, "deviation-heading": 6.302288303368849, "agent_compute-ego0": 0.013186906199967437, "complete-iteration": 0.16328562328360857, "set_robot_commands": 0.0023779652696366512, "deviation-center-line": 3.917121392328863, "driven_lanedir_consec": 7.167994682911328, "sim_compute_sim_state": 0.006524714899499847, "sim_compute_performance-ego0": 0.0021788720584332596}}
set_robot_commands_max0.0023779652696366512
set_robot_commands_mean0.0023006953151625763
set_robot_commands_median0.0023270416176562024
set_robot_commands_min0.00217073275570125
sim_compute_performance-ego0_max0.0021788720584332596
sim_compute_performance-ego0_mean0.0020944745629210633
sim_compute_performance-ego0_median0.0021152081963823517
sim_compute_performance-ego0_min0.0019686098004862897
sim_compute_sim_state_max0.012064422736188027
sim_compute_sim_state_mean0.009851172247618216
sim_compute_sim_state_median0.010407775677392494
sim_compute_sim_state_min0.006524714899499847
sim_render-ego0_max0.004113445869591909
sim_render-ego0_mean0.0039912568706474815
sim_render-ego0_median0.004011099149712998
sim_render-ego0_min0.003829383313572021
simulation-passed1
step_physics_max0.141489428502542
step_physics_mean0.10985118267056634
step_physics_median0.1012133331075272
step_physics_min0.095488635964669
survival_time_max59.99999999999873
survival_time_mean29.312499999999737
survival_time_min5.349999999999989
No reset possible
5734511426Frank (Chude) QianΒ πŸ‡¨πŸ‡¦template-pytorchaido5-LFP-sim-testingLFP-simhost-erroryesnogpu-prod-030:00:43
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5728310970Fernanda Custodio Pereira do CarmoΒ πŸ‡¨πŸ‡¦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-030:14:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median2.0763814660382116
survival_time_median16.075000000000095
deviation-center-line_median0.8623462757520712
in-drivable-lane_median4.325000000000019


other stats
agent_compute-ego0_max0.013224216246269118
agent_compute-ego0_mean0.012758087760683406
agent_compute-ego0_median0.012694971450483835
agent_compute-ego0_min0.012418191895496837
complete-iteration_max0.21564229715217664
complete-iteration_mean0.17935230044405295
complete-iteration_median0.17331022913447858
complete-iteration_min0.15514644635507804
deviation-center-line_max4.226451893811083
deviation-center-line_mean1.522205672926186
deviation-center-line_min0.13767824638951892
deviation-heading_max5.6256776107874265
deviation-heading_mean2.6457951439595098
deviation-heading_median2.050805061758666
deviation-heading_min0.8558928415332797
driven_any_max10.672712226884848
driven_any_mean4.193993794866213
driven_any_median2.688479599482527
driven_any_min0.7263037536149477
driven_lanedir_consec_max6.565569938508501
driven_lanedir_consec_mean2.7424637079988425
driven_lanedir_consec_min0.2515219614104458
driven_lanedir_max6.565569938508501
driven_lanedir_mean2.7424637079988425
driven_lanedir_median2.0763814660382116
driven_lanedir_min0.2515219614104458
get_duckie_state_max2.1802967992322196e-06
get_duckie_state_mean2.0961459431249243e-06
get_duckie_state_median2.089368226675552e-06
get_duckie_state_min2.025550519916373e-06
get_robot_state_max0.0037418324371864056
get_robot_state_mean0.0036560442320966017
get_robot_state_median0.0036665162831466935
get_robot_state_min0.0035493119249066104
get_state_dump_max0.00470274267062335
get_state_dump_mean0.004589371676233317
get_state_dump_median0.004556010295782818
get_state_dump_min0.0045427234427442825
get_ui_image_max0.03481195505382945
get_ui_image_mean0.030329287861849703
get_ui_image_median0.030183975999942388
get_ui_image_min0.026137244393684585
in-drivable-lane_max22.299999999999493
in-drivable-lane_mean8.26249999999989
in-drivable-lane_min2.10000000000003
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 2.4032958652248504, "get_ui_image": 0.027682706405376567, "step_physics": 0.0944355167191604, "survival_time": 14.45000000000007, "driven_lanedir": 2.1206971359687383, "get_state_dump": 0.004568483089578562, "get_robot_state": 0.0037418324371864056, "sim_render-ego0": 0.003940780409451189, "get_duckie_state": 2.1802967992322196e-06, "in-drivable-lane": 2.10000000000003, "deviation-heading": 1.5199961348902955, "agent_compute-ego0": 0.012614889802603889, "complete-iteration": 0.1610605165876191, "set_robot_commands": 0.0022908761583525557, "deviation-center-line": 0.5943880053355016, "driven_lanedir_consec": 2.1206971359687383, "sim_compute_sim_state": 0.00970078418994772, "sim_compute_performance-ego0": 0.0019965969283005287}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.7263037536149477, "get_ui_image": 0.03481195505382945, "step_physics": 0.1421579527623445, "survival_time": 5.09999999999999, "driven_lanedir": 0.2515219614104458, "get_state_dump": 0.0045427234427442825, "get_robot_state": 0.0035493119249066104, "sim_render-ego0": 0.003853200708778159, "get_duckie_state": 2.099472342185604e-06, "in-drivable-lane": 3.1999999999999886, "deviation-heading": 0.8558928415332797, "agent_compute-ego0": 0.012775053098363784, "complete-iteration": 0.21564229715217664, "set_robot_commands": 0.0022101957821151585, "deviation-center-line": 0.13767824638951892, "driven_lanedir_consec": 0.2515219614104458, "sim_compute_sim_state": 0.009721598578888236, "sim_compute_performance-ego0": 0.001933077006664091}, "LF-norm-techtrack-000-ego0": {"driven_any": 2.9736633337402036, "get_ui_image": 0.03268524559450821, "step_physics": 0.11211776666238274, "survival_time": 17.700000000000117, "driven_lanedir": 2.032065796107685, "get_state_dump": 0.00470274267062335, "get_robot_state": 0.003730036507190113, "sim_render-ego0": 0.003981263872603296, "get_duckie_state": 2.025550519916373e-06, "in-drivable-lane": 5.450000000000048, "deviation-heading": 2.581613988627037, "agent_compute-ego0": 0.013224216246269118, "complete-iteration": 0.185559941681338, "set_robot_commands": 0.002181876545221034, "deviation-center-line": 1.130304546168641, "driven_lanedir_consec": 2.032065796107685, "sim_compute_sim_state": 0.010718003125257896, "sim_compute_performance-ego0": 0.002131724021804165}, "LF-norm-small_loop-000-ego0": {"driven_any": 10.672712226884848, "get_ui_image": 0.026137244393684585, "step_physics": 0.09434215651265192, "survival_time": 59.99999999999873, "driven_lanedir": 6.565569938508501, "get_state_dump": 0.004543537501987073, "get_robot_state": 0.003602996059103274, "sim_render-ego0": 0.0038438929209204937, "get_duckie_state": 2.0792641111655e-06, "in-drivable-lane": 22.299999999999493, "deviation-heading": 5.6256776107874265, "agent_compute-ego0": 0.012418191895496837, "complete-iteration": 0.15514644635507804, "set_robot_commands": 0.002195264179442546, "deviation-center-line": 4.226451893811083, "driven_lanedir_consec": 6.565569938508501, "sim_compute_sim_state": 0.006036303025498974, "sim_compute_performance-ego0": 0.00194114689822995}}
set_robot_commands_max0.0022908761583525557
set_robot_commands_mean0.002219553166282823
set_robot_commands_median0.0022027299807788523
set_robot_commands_min0.002181876545221034
sim_compute_performance-ego0_max0.002131724021804165
sim_compute_performance-ego0_mean0.002000636213749684
sim_compute_performance-ego0_median0.001968871913265239
sim_compute_performance-ego0_min0.001933077006664091
sim_compute_sim_state_max0.010718003125257896
sim_compute_sim_state_mean0.009044172229898204
sim_compute_sim_state_median0.009711191384417978
sim_compute_sim_state_min0.006036303025498974
sim_render-ego0_max0.003981263872603296
sim_render-ego0_mean0.003904784477938284
sim_render-ego0_median0.003896990559114674
sim_render-ego0_min0.0038438929209204937
simulation-passed1
step_physics_max0.1421579527623445
step_physics_mean0.1107633481641349
step_physics_median0.10327664169077158
step_physics_min0.09434215651265192
survival_time_max59.99999999999873
survival_time_mean24.312499999999726
survival_time_min5.09999999999999
No reset possible
5726611313Moustafa Elarabitemplate-pytorchaido5-LFV_multi-sim-validationLFVmultibodyv-simhost-erroryesnogpu-prod-030:01:12
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5724011010Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simabortedyesnogpu-prod-030:02:29
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 45, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 45, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 45, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 45, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5721410992Ayman ShamsΒ πŸ‡¨πŸ‡¦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-030:12:05
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.5041387925120766
survival_time_median19.475000000000144
deviation-center-line_median0.3209047358386513
in-drivable-lane_median14.9750000000001


other stats
agent_compute-ego0_max0.01422380014907482
agent_compute-ego0_mean0.013949512780180933
agent_compute-ego0_median0.014039806686716498
agent_compute-ego0_min0.013494637598215916
complete-iteration_max0.2607944593873135
complete-iteration_mean0.2016787956251908
complete-iteration_median0.18540021234411536
complete-iteration_min0.17512029842521906
deviation-center-line_max0.5396974418934898
deviation-center-line_mean0.3208466319927403
deviation-center-line_min0.1018796144001689
deviation-heading_max4.562750794687354
deviation-heading_mean2.132655456331158
deviation-heading_median1.7030302658345668
deviation-heading_min0.5618104989681448
driven_any_max5.145297860596097
driven_any_mean3.1858074355045667
driven_any_median3.487395664857684
driven_any_min0.6231405517068018
driven_lanedir_consec_max0.774651655368454
driven_lanedir_consec_mean0.4953696592245167
driven_lanedir_consec_min0.19854939650545944
driven_lanedir_max0.774651655368454
driven_lanedir_mean0.4953696592245167
driven_lanedir_median0.5041387925120766
driven_lanedir_min0.19854939650545944
get_duckie_state_max1.8269328747765493e-06
get_duckie_state_mean1.6314424448862957e-06
get_duckie_state_median1.608175914935341e-06
get_duckie_state_min1.482485074897951e-06
get_robot_state_max0.004165300103121026
get_robot_state_mean0.004072623498115268
get_robot_state_median0.004059393730836415
get_robot_state_min0.004006406427667216
get_state_dump_max0.005350083819890427
get_state_dump_mean0.005172691208871384
get_state_dump_median0.005208041282970438
get_state_dump_min0.004924598449654233
get_ui_image_max0.039682546327280445
get_ui_image_mean0.03300349241499543
get_ui_image_median0.03220159777419469
get_ui_image_min0.027928227784311897
in-drivable-lane_max22.700000000000216
in-drivable-lane_mean13.8000000000001
in-drivable-lane_min2.549999999999992
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 4.320744844122862, "get_ui_image": 0.030272952424801464, "step_physics": 0.10076138886009776, "survival_time": 24.550000000000217, "driven_lanedir": 0.774651655368454, "get_state_dump": 0.005133170422499742, "get_robot_state": 0.00408613197202605, "sim_render-ego0": 0.00433008990636686, "get_duckie_state": 1.619501811702077e-06, "in-drivable-lane": 17.35000000000013, "deviation-heading": 2.4991226826903397, "agent_compute-ego0": 0.014108736825183156, "complete-iteration": 0.17527399169720284, "set_robot_commands": 0.002488075717677915, "deviation-center-line": 0.5396974418934898, "driven_lanedir_consec": 0.774651655368454, "sim_compute_sim_state": 0.011746461313914476, "sim_compute_performance-ego0": 0.0022419104731179833}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.6231405517068018, "get_ui_image": 0.039682546327280445, "step_physics": 0.1768503050471461, "survival_time": 4.249999999999993, "driven_lanedir": 0.19854939650545944, "get_state_dump": 0.005282912143441134, "get_robot_state": 0.004165300103121026, "sim_render-ego0": 0.004366952319477879, "get_duckie_state": 1.5968500181686047e-06, "in-drivable-lane": 2.549999999999992, "deviation-heading": 0.906937848978794, "agent_compute-ego0": 0.01422380014907482, "complete-iteration": 0.2607944593873135, "set_robot_commands": 0.002468480620273324, "deviation-center-line": 0.1018796144001689, "driven_lanedir_consec": 0.19854939650545944, "sim_compute_sim_state": 0.011442250983659611, "sim_compute_performance-ego0": 0.002209192098573197}, "LF-norm-techtrack-000-ego0": {"driven_any": 5.145297860596097, "get_ui_image": 0.034130243123587915, "step_physics": 0.11600111581511416, "survival_time": 29.450000000000284, "driven_lanedir": 0.7327203204935485, "get_state_dump": 0.005350083819890427, "get_robot_state": 0.004032655489646782, "sim_render-ego0": 0.004340749271845414, "get_duckie_state": 1.8269328747765493e-06, "in-drivable-lane": 22.700000000000216, "deviation-heading": 4.562750794687354, "agent_compute-ego0": 0.013970876548249843, "complete-iteration": 0.19552643299102784, "set_robot_commands": 0.00247255018201925, "deviation-center-line": 0.5314351558339371, "driven_lanedir_consec": 0.7327203204935485, "sim_compute_sim_state": 0.01283081507278701, "sim_compute_performance-ego0": 0.0022825192596952794}, "LF-norm-small_loop-000-ego0": {"driven_any": 2.6540464855925054, "get_ui_image": 0.027928227784311897, "step_physics": 0.10986298838288726, "survival_time": 14.40000000000007, "driven_lanedir": 0.2755572645306048, "get_state_dump": 0.004924598449654233, "get_robot_state": 0.004006406427667216, "sim_render-ego0": 0.004105774176574496, "get_duckie_state": 1.482485074897951e-06, "in-drivable-lane": 12.600000000000067, "deviation-heading": 0.5618104989681448, "agent_compute-ego0": 0.013494637598215916, "complete-iteration": 0.17512029842521906, "set_robot_commands": 0.0024038656360137836, "deviation-center-line": 0.11037431584336552, "driven_lanedir_consec": 0.2755572645306048, "sim_compute_sim_state": 0.006132035931913911, "sim_compute_performance-ego0": 0.002163217966944289}}
set_robot_commands_max0.002488075717677915
set_robot_commands_mean0.002458243038996068
set_robot_commands_median0.0024705154011462867
set_robot_commands_min0.0024038656360137836
sim_compute_performance-ego0_max0.0022825192596952794
sim_compute_performance-ego0_mean0.002224209949582687
sim_compute_performance-ego0_median0.0022255512858455903
sim_compute_performance-ego0_min0.002163217966944289
sim_compute_sim_state_max0.01283081507278701
sim_compute_sim_state_mean0.010537890825568752
sim_compute_sim_state_median0.011594356148787044
sim_compute_sim_state_min0.006132035931913911
sim_render-ego0_max0.004366952319477879
sim_render-ego0_mean0.004285891418566163
sim_render-ego0_median0.004335419589106137
sim_render-ego0_min0.004105774176574496
simulation-passed1
step_physics_max0.1768503050471461
step_physics_mean0.1258689495263113
step_physics_median0.11293205209900072
step_physics_min0.10076138886009776
survival_time_max29.450000000000284
survival_time_mean18.16250000000014
survival_time_min4.249999999999993
No reset possible
5720511012Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simabortedyesnogpu-prod-030:02:51
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 45, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 45, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 45, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 45, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5707011025Mo KleitΒ πŸ‡¨πŸ‡¦sim-exercise-1aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-030:22:26
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.9753387802844635
survival_time_median38.64999999999942
deviation-center-line_median1.2319156234441822
in-drivable-lane_median20.774999999999665


other stats
agent_compute-ego0_max0.03335689485916786
agent_compute-ego0_mean0.01830228583566683
agent_compute-ego0_median0.013651538476647657
agent_compute-ego0_min0.012549171530204135
complete-iteration_max0.19771311408595035
complete-iteration_mean0.1756350605960873
complete-iteration_median0.16963121036209883
complete-iteration_min0.16556470757420116
deviation-center-line_max2.078859203778461
deviation-center-line_mean1.2019123670580194
deviation-center-line_min0.2649590175652525
deviation-heading_max8.072051478664207
deviation-heading_mean4.408473049121105
deviation-heading_median3.95361182358318
deviation-heading_min1.6546170706538528
driven_any_max7.678237257813193
driven_any_mean4.047420184892982
driven_any_median3.950597712810248
driven_any_min0.6102480561382378
driven_lanedir_consec_max5.382364798943632
driven_lanedir_consec_mean2.3873258806860465
driven_lanedir_consec_min0.2162611632316271
driven_lanedir_max5.382364798943632
driven_lanedir_mean2.3873258806860465
driven_lanedir_median1.9753387802844635
driven_lanedir_min0.2162611632316271
get_duckie_state_max1.3853374280427632e-06
get_duckie_state_mean1.2803014553599396e-06
get_duckie_state_median1.2622685555514449e-06
get_duckie_state_min1.2113312822941056e-06
get_robot_state_max0.003782306219402112
get_robot_state_mean0.0036260707890494945
get_robot_state_median0.003610206980391605
get_robot_state_min0.003501562976012656
get_state_dump_max0.004707709111665424
get_state_dump_mean0.004592803411553532
get_state_dump_median0.00459297054713215
get_state_dump_min0.0044775634402844
get_ui_image_max0.0369784016358225
get_ui_image_mean0.030239442228859013
get_ui_image_median0.02947859564729982
get_ui_image_min0.02502217598501391
in-drivable-lane_max39.199999999998525
in-drivable-lane_mean21.649999999999466
in-drivable-lane_min5.850000000000004
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 1.7670270063533613, "get_ui_image": 0.02716123061496174, "step_physics": 0.10437201835236563, "survival_time": 17.30000000000011, "driven_lanedir": 0.8881247103472769, "get_state_dump": 0.0044775634402844, "get_robot_state": 0.003501562976012656, "sim_render-ego0": 0.0038133938649202287, "get_duckie_state": 1.2113312822941056e-06, "in-drivable-lane": 12.60000000000012, "deviation-heading": 1.6546170706538528, "agent_compute-ego0": 0.012549171530204135, "complete-iteration": 0.1689416359068681, "set_robot_commands": 0.0021068871193042063, "deviation-center-line": 0.5084008713984072, "driven_lanedir_consec": 0.8881247103472769, "sim_compute_sim_state": 0.009016523443656284, "sim_compute_performance-ego0": 0.0018625637296297364}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.6102480561382378, "get_ui_image": 0.0369784016358225, "step_physics": 0.11832766909348336, "survival_time": 9.45, "driven_lanedir": 0.2162611632316271, "get_state_dump": 0.004707709111665424, "get_robot_state": 0.003782306219402112, "sim_render-ego0": 0.004134749111376311, "get_duckie_state": 1.3853374280427632e-06, "in-drivable-lane": 5.850000000000004, "deviation-heading": 1.8665402725114908, "agent_compute-ego0": 0.014113876694127132, "complete-iteration": 0.19771311408595035, "set_robot_commands": 0.0023339296642102695, "deviation-center-line": 0.2649590175652525, "driven_lanedir_consec": 0.2162611632316271, "sim_compute_sim_state": 0.011181747285943282, "sim_compute_performance-ego0": 0.002062921775014777}, "LF-norm-techtrack-000-ego0": {"driven_any": 7.678237257813193, "get_ui_image": 0.0317959606796379, "step_physics": 0.0986778364888238, "survival_time": 59.99999999999873, "driven_lanedir": 5.382364798943632, "get_state_dump": 0.0045084059982871535, "get_robot_state": 0.0036438791876927104, "sim_render-ego0": 0.003910091298505924, "get_duckie_state": 1.2121430840917074e-06, "in-drivable-lane": 28.94999999999921, "deviation-heading": 8.072051478664207, "agent_compute-ego0": 0.013189200259168182, "complete-iteration": 0.17032078481732954, "set_robot_commands": 0.0021672373905864783, "deviation-center-line": 1.9554303754899571, "driven_lanedir_consec": 5.382364798943632, "sim_compute_sim_state": 0.010357544285173124, "sim_compute_performance-ego0": 0.0019880802208537564}, "LF-norm-small_loop-000-ego0": {"driven_any": 6.134168419267135, "get_ui_image": 0.02502217598501391, "step_physics": 0.08503971925683065, "survival_time": 59.99999999999873, "driven_lanedir": 3.0625528502216506, "get_state_dump": 0.004677535095977148, "get_robot_state": 0.0035765347730904993, "sim_render-ego0": 0.003734145533730843, "get_duckie_state": 1.312394027011182e-06, "in-drivable-lane": 39.199999999998525, "deviation-heading": 6.040683374654869, "agent_compute-ego0": 0.03335689485916786, "complete-iteration": 0.16556470757420116, "set_robot_commands": 0.002148088070871828, "deviation-center-line": 2.078859203778461, "driven_lanedir_consec": 3.0625528502216506, "sim_compute_sim_state": 0.006053125729271018, "sim_compute_performance-ego0": 0.0018704215453923692}}
set_robot_commands_max0.0023339296642102695
set_robot_commands_mean0.0021890355612431953
set_robot_commands_median0.0021576627307291533
set_robot_commands_min0.0021068871193042063
sim_compute_performance-ego0_max0.002062921775014777
sim_compute_performance-ego0_mean0.0019459968177226595
sim_compute_performance-ego0_median0.001929250883123063
sim_compute_performance-ego0_min0.0018625637296297364
sim_compute_sim_state_max0.011181747285943282
sim_compute_sim_state_mean0.009152235186010929
sim_compute_sim_state_median0.009687033864414704
sim_compute_sim_state_min0.006053125729271018
sim_render-ego0_max0.004134749111376311
sim_render-ego0_mean0.0038980949521333266
sim_render-ego0_median0.0038617425817130766
sim_render-ego0_min0.003734145533730843
simulation-passed1
step_physics_max0.11832766909348336
step_physics_mean0.10160431079787584
step_physics_median0.10152492742059473
step_physics_min0.08503971925683065
survival_time_max59.99999999999873
survival_time_mean36.68749999999939
survival_time_min9.45
No reset possible
5705411312Moustafa Elarabitemplate-pytorchaido5-LFP-sim-validationLFP-simhost-errornonogpu-prod-030:02:25
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5703311427Frank (Chude) QianΒ πŸ‡¨πŸ‡¦template-pytorchaido5-LFV_multi-sim-validationLFVmultibodyv-simhost-erroryesnogpu-prod-030:02:40
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5649911416Frank (Chude) QianΒ πŸ‡¨πŸ‡¦challenge-aido_LF-baseline-behavior-cloningaido5-LFV_multi-sim-testingLFVmultibodyv-simsuccessyesnogpu-prod-031:31:00
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median6.527457152398652
survival_time_median59.99999999999873
deviation-center-line_median2.566012465313792
in-drivable-lane_median2.974999999999917


other stats
agent_compute-ego0_max0.043790828000317826
agent_compute-ego0_mean0.04273580824536958
agent_compute-ego0_median0.0425901730590617
agent_compute-ego0_min0.04150709739098182
agent_compute-ego1_max0.043272539839014273
agent_compute-ego1_mean0.042703167027205176
agent_compute-ego1_median0.04256571818152435
agent_compute-ego1_min0.042300931289005424
complete-iteration_max1.2033229360572504
complete-iteration_mean0.8396254145275922
complete-iteration_median0.8020484699859275
complete-iteration_min0.34394260697055545
deviation-center-line_max7.095212259829976
deviation-center-line_mean2.5251564033814455
deviation-center-line_min0.17012276741346194
deviation-heading_max22.14669411830537
deviation-heading_mean10.090350984645085
deviation-heading_median10.62680681832336
deviation-heading_min1.0191473292537532
driven_any_max14.590134540077422
driven_any_mean8.712047713743372
driven_any_median7.840342262593758
driven_any_min0.5526827992852295
driven_lanedir_consec_max13.490777819275216
driven_lanedir_consec_mean7.551919798756596
driven_lanedir_consec_min0.4801949691561933
driven_lanedir_max13.490777819275216
driven_lanedir_mean7.551919798756596
driven_lanedir_median6.527457152398652
driven_lanedir_min0.4801949691561933
get_duckie_state_max1.5093226119143878e-06
get_duckie_state_mean1.4133799084947776e-06
get_duckie_state_median1.5013926738017314e-06
get_duckie_state_min1.274080300311264e-06
get_robot_state_max0.014326535366676949
get_robot_state_mean0.012938580518386292
get_robot_state_median0.013950311571831112
get_robot_state_min0.007254326145753242
get_state_dump_max0.009843729637764595
get_state_dump_mean0.00884774880640716
get_state_dump_median0.009218672332318994
get_state_dump_min0.006308674272908345
get_ui_image_max0.0474674054526171
get_ui_image_mean0.04236380397620383
get_ui_image_median0.04324476427083011
get_ui_image_min0.029869042190850948
in-drivable-lane_max57.44999999999872
in-drivable-lane_mean7.221428571428437
in-drivable-lane_min0.0
per-episodes
details{"LFV_multi-norm-loop-000-ego0": {"driven_any": 5.162530616459116, "get_ui_image": 0.04262662309784073, "step_physics": 0.5031998565605095, "survival_time": 22.15000000000018, "driven_lanedir": 4.585138479803474, "get_state_dump": 0.009843729637764595, "get_robot_state": 0.014326535366676949, "sim_render-ego0": 0.003997539614771938, "sim_render-ego1": 0.004061796643712499, "sim_render-ego2": 0.0039258534843857225, "sim_render-ego3": 0.003916215789210689, "get_duckie_state": 1.5013926738017314e-06, "in-drivable-lane": 2.0000000000000133, "deviation-heading": 5.080848239674109, "agent_compute-ego0": 0.043790828000317826, "agent_compute-ego1": 0.043272539839014273, "agent_compute-ego2": 0.04314277247265653, "agent_compute-ego3": 0.04433220839715219, "complete-iteration": 0.8020484699859275, "set_robot_commands": 0.0022761166632712425, "deviation-center-line": 1.2097105748276284, "driven_lanedir_consec": 4.585138479803474, "sim_compute_sim_state": 0.023889723124804797, "sim_compute_performance-ego0": 0.002087103353964316, "sim_compute_performance-ego1": 0.002017368067492236, "sim_compute_performance-ego2": 0.0019963738080617543, "sim_compute_performance-ego3": 0.0020274106446687167}, "LFV_multi-norm-loop-000-ego1": {"driven_any": 3.450656015667263, "get_ui_image": 0.04262662309784073, "step_physics": 0.5031998565605095, "survival_time": 22.15000000000018, "driven_lanedir": 3.1722393416603047, "get_state_dump": 0.009843729637764595, "get_robot_state": 0.014326535366676949, "sim_render-ego0": 0.003997539614771938, "sim_render-ego1": 0.004061796643712499, "sim_render-ego2": 0.0039258534843857225, "sim_render-ego3": 0.003916215789210689, "get_duckie_state": 1.5013926738017314e-06, "in-drivable-lane": 0.6499999999999977, "deviation-heading": 3.0451611783649097, "agent_compute-ego0": 0.043790828000317826, "agent_compute-ego1": 0.043272539839014273, "agent_compute-ego2": 0.04314277247265653, "agent_compute-ego3": 0.04433220839715219, "complete-iteration": 0.8020484699859275, "set_robot_commands": 0.0022761166632712425, "deviation-center-line": 1.3987767505060926, "driven_lanedir_consec": 3.1722393416603047, "sim_compute_sim_state": 0.023889723124804797, "sim_compute_performance-ego0": 0.002087103353964316, "sim_compute_performance-ego1": 0.002017368067492236, "sim_compute_performance-ego2": 0.0019963738080617543, "sim_compute_performance-ego3": 0.0020274106446687167}, "LFV_multi-norm-loop-000-ego2": {"driven_any": 3.450690726278296, "get_ui_image": 0.04262662309784073, "step_physics": 0.5031998565605095, "survival_time": 22.15000000000018, "driven_lanedir": 3.098969449214533, "get_state_dump": 0.009843729637764595, "get_robot_state": 0.014326535366676949, "sim_render-ego0": 0.003997539614771938, "sim_render-ego1": 0.004061796643712499, "sim_render-ego2": 0.0039258534843857225, "sim_render-ego3": 0.003916215789210689, "get_duckie_state": 1.5013926738017314e-06, "in-drivable-lane": 1.099999999999996, "deviation-heading": 7.479417725018329, "agent_compute-ego0": 0.043790828000317826, "agent_compute-ego1": 0.043272539839014273, "agent_compute-ego2": 0.04314277247265653, "agent_compute-ego3": 0.04433220839715219, "complete-iteration": 0.8020484699859275, "set_robot_commands": 0.0022761166632712425, "deviation-center-line": 1.2555413745240371, "driven_lanedir_consec": 3.098969449214533, "sim_compute_sim_state": 0.023889723124804797, "sim_compute_performance-ego0": 0.002087103353964316, "sim_compute_performance-ego1": 0.002017368067492236, "sim_compute_performance-ego2": 0.0019963738080617543, "sim_compute_performance-ego3": 0.0020274106446687167}, "LFV_multi-norm-loop-000-ego3": {"driven_any": 5.162641684723338, "get_ui_image": 0.04262662309784073, "step_physics": 0.5031998565605095, "survival_time": 22.15000000000018, "driven_lanedir": 5.106283806909655, "get_state_dump": 0.009843729637764595, "get_robot_state": 0.014326535366676949, "sim_render-ego0": 0.003997539614771938, "sim_render-ego1": 0.004061796643712499, "sim_render-ego2": 0.0039258534843857225, "sim_render-ego3": 0.003916215789210689, "get_duckie_state": 1.5013926738017314e-06, "in-drivable-lane": 0.0, "deviation-heading": 2.5588895069566258, "agent_compute-ego0": 0.043790828000317826, "agent_compute-ego1": 0.043272539839014273, "agent_compute-ego2": 0.04314277247265653, "agent_compute-ego3": 0.04433220839715219, "complete-iteration": 0.8020484699859275, "set_robot_commands": 0.0022761166632712425, "deviation-center-line": 0.8373073979025072, "driven_lanedir_consec": 5.106283806909655, "sim_compute_sim_state": 0.023889723124804797, "sim_compute_performance-ego0": 0.002087103353964316, "sim_compute_performance-ego1": 0.002017368067492236, "sim_compute_performance-ego2": 0.0019963738080617543, "sim_compute_performance-ego3": 0.0020274106446687167}, "LFV_multi-norm-zigzag-000-ego0": {"driven_any": 0.5526827992852295, "get_ui_image": 0.0474674054526171, "step_physics": 0.904519536116042, "survival_time": 59.99999999999873, "driven_lanedir": 0.4801949691561933, "get_state_dump": 0.009218672332318994, "get_robot_state": 0.013950311571831112, "sim_render-ego0": 0.003931518994600549, "sim_render-ego1": 0.003837694831930727, "sim_render-ego2": 0.0037561096219992656, "sim_render-ego3": 0.00377320330109227, "get_duckie_state": 1.5093226119143878e-06, "in-drivable-lane": 57.44999999999872, "deviation-heading": 1.0191473292537532, "agent_compute-ego0": 0.0425901730590617, "agent_compute-ego1": 0.04256571818152435, "agent_compute-ego2": 0.04117860960821426, "agent_compute-ego3": 0.04303795709697333, "complete-iteration": 1.2033229360572504, "set_robot_commands": 0.0021767346289235288, "deviation-center-line": 0.17012276741346194, "driven_lanedir_consec": 0.4801949691561933, "sim_compute_sim_state": 0.026022616075933425, "sim_compute_performance-ego0": 0.002044381348914052, "sim_compute_performance-ego1": 0.0020005329760981044, "sim_compute_performance-ego2": 0.0019101715405517377, "sim_compute_performance-ego3": 0.0019405704056789832}, "LFV_multi-norm-zigzag-000-ego1": {"driven_any": 14.590080756858097, "get_ui_image": 0.0474674054526171, "step_physics": 0.904519536116042, "survival_time": 59.99999999999873, "driven_lanedir": 12.882676879579952, "get_state_dump": 0.009218672332318994, "get_robot_state": 0.013950311571831112, "sim_render-ego0": 0.003931518994600549, "sim_render-ego1": 0.003837694831930727, "sim_render-ego2": 0.0037561096219992656, "sim_render-ego3": 0.00377320330109227, "get_duckie_state": 1.5093226119143878e-06, "in-drivable-lane": 3.849999999999907, "deviation-heading": 14.323086911273473, "agent_compute-ego0": 0.0425901730590617, "agent_compute-ego1": 0.04256571818152435, "agent_compute-ego2": 0.04117860960821426, "agent_compute-ego3": 0.04303795709697333, "complete-iteration": 1.2033229360572504, "set_robot_commands": 0.0021767346289235288, "deviation-center-line": 3.382410185031243, "driven_lanedir_consec": 12.882676879579952, "sim_compute_sim_state": 0.026022616075933425, "sim_compute_performance-ego0": 0.002044381348914052, "sim_compute_performance-ego1": 0.0020005329760981044, "sim_compute_performance-ego2": 0.0019101715405517377, "sim_compute_performance-ego3": 0.0019405704056789832}, "LFV_multi-norm-zigzag-000-ego2": {"driven_any": 0.9683871442495025, "get_ui_image": 0.0474674054526171, "step_physics": 0.904519536116042, "survival_time": 59.99999999999873, "driven_lanedir": 0.5967237345825644, "get_state_dump": 0.009218672332318994, "get_robot_state": 0.013950311571831112, "sim_render-ego0": 0.003931518994600549, "sim_render-ego1": 0.003837694831930727, "sim_render-ego2": 0.0037561096219992656, "sim_render-ego3": 0.00377320330109227, "get_duckie_state": 1.5093226119143878e-06, "in-drivable-lane": 1.0499999999999996, "deviation-heading": 22.14669411830537, "agent_compute-ego0": 0.0425901730590617, "agent_compute-ego1": 0.04256571818152435, "agent_compute-ego2": 0.04117860960821426, "agent_compute-ego3": 0.04303795709697333, "complete-iteration": 1.2033229360572504, "set_robot_commands": 0.0021767346289235288, "deviation-center-line": 7.095212259829976, "driven_lanedir_consec": 0.5967237345825644, "sim_compute_sim_state": 0.026022616075933425, "sim_compute_performance-ego0": 0.002044381348914052, "sim_compute_performance-ego1": 0.0020005329760981044, "sim_compute_performance-ego2": 0.0019101715405517377, "sim_compute_performance-ego3": 0.0019405704056789832}, "LFV_multi-norm-zigzag-000-ego3": {"driven_any": 14.590134540077422, "get_ui_image": 0.0474674054526171, "step_physics": 0.904519536116042, "survival_time": 59.99999999999873, "driven_lanedir": 13.490777819275216, "get_state_dump": 0.009218672332318994, "get_robot_state": 0.013950311571831112, "sim_render-ego0": 0.003931518994600549, "sim_render-ego1": 0.003837694831930727, "sim_render-ego2": 0.0037561096219992656, "sim_render-ego3": 0.00377320330109227, "get_duckie_state": 1.5093226119143878e-06, "in-drivable-lane": 2.099999999999927, "deviation-heading": 12.863706476516798, "agent_compute-ego0": 0.0425901730590617, "agent_compute-ego1": 0.04256571818152435, "agent_compute-ego2": 0.04117860960821426, "agent_compute-ego3": 0.04303795709697333, "complete-iteration": 1.2033229360572504, "set_robot_commands": 0.0021767346289235288, "deviation-center-line": 3.157798543097953, "driven_lanedir_consec": 13.490777819275216, "sim_compute_sim_state": 0.026022616075933425, "sim_compute_performance-ego0": 0.002044381348914052, "sim_compute_performance-ego1": 0.0020005329760981044, "sim_compute_performance-ego2": 0.0019101715405517377, "sim_compute_performance-ego3": 0.0019405704056789832}, "LFV_multi-norm-techtrack-000-ego0": {"driven_any": 14.59005892183093, "get_ui_image": 0.04324476427083011, "step_physics": 0.4586728026130416, "survival_time": 59.99999999999873, "driven_lanedir": 12.397963702635836, "get_state_dump": 0.008750381715887294, "get_robot_state": 0.013381021802967337, "sim_render-ego0": 0.0035744221581706, "sim_render-ego1": 0.0035646535474791515, "sim_render-ego2": 0.003510539279591531, "sim_render-ego3": 0.003519218430531015, "get_duckie_state": 1.274080300311264e-06, "in-drivable-lane": 5.749999999999865, "deviation-heading": 14.222718825506938, "agent_compute-ego0": 0.042440779103923104, "agent_compute-ego1": 0.0424723609301768, "agent_compute-ego2": 0.04153643301583448, "agent_compute-ego3": 0.0422902015921079, "complete-iteration": 0.7613462413181175, "set_robot_commands": 0.002008956834537401, "deviation-center-line": 3.3199897435404275, "driven_lanedir_consec": 12.397963702635836, "sim_compute_sim_state": 0.038915868603518165, "sim_compute_performance-ego0": 0.00188107812136635, "sim_compute_performance-ego1": 0.0017921068984007062, "sim_compute_performance-ego2": 0.001755967922353625, "sim_compute_performance-ego3": 0.001773173961909387}, "LFV_multi-norm-techtrack-000-ego1": {"driven_any": 14.5900520938242, "get_ui_image": 0.04324476427083011, "step_physics": 0.4586728026130416, "survival_time": 59.99999999999873, "driven_lanedir": 12.2852036922114, "get_state_dump": 0.008750381715887294, "get_robot_state": 0.013381021802967337, "sim_render-ego0": 0.0035744221581706, "sim_render-ego1": 0.0035646535474791515, "sim_render-ego2": 0.003510539279591531, "sim_render-ego3": 0.003519218430531015, "get_duckie_state": 1.274080300311264e-06, "in-drivable-lane": 5.949999999999918, "deviation-heading": 15.542743057819589, "agent_compute-ego0": 0.042440779103923104, "agent_compute-ego1": 0.0424723609301768, "agent_compute-ego2": 0.04153643301583448, "agent_compute-ego3": 0.0422902015921079, "complete-iteration": 0.7613462413181175, "set_robot_commands": 0.002008956834537401, "deviation-center-line": 3.446257707159583, "driven_lanedir_consec": 12.2852036922114, "sim_compute_sim_state": 0.038915868603518165, "sim_compute_performance-ego0": 0.00188107812136635, "sim_compute_performance-ego1": 0.0017921068984007062, "sim_compute_performance-ego2": 0.001755967922353625, "sim_compute_performance-ego3": 0.001773173961909387}, "LFV_multi-norm-techtrack-000-ego2": {"driven_any": 14.589996332905445, "get_ui_image": 0.04324476427083011, "step_physics": 0.4586728026130416, "survival_time": 59.99999999999873, "driven_lanedir": 12.20064691564406, "get_state_dump": 0.008750381715887294, "get_robot_state": 0.013381021802967337, "sim_render-ego0": 0.0035744221581706, "sim_render-ego1": 0.0035646535474791515, "sim_render-ego2": 0.003510539279591531, "sim_render-ego3": 0.003519218430531015, "get_duckie_state": 1.274080300311264e-06, "in-drivable-lane": 6.199999999999857, "deviation-heading": 15.638557852834548, "agent_compute-ego0": 0.042440779103923104, "agent_compute-ego1": 0.0424723609301768, "agent_compute-ego2": 0.04153643301583448, "agent_compute-ego3": 0.0422902015921079, "complete-iteration": 0.7613462413181175, "set_robot_commands": 0.002008956834537401, "deviation-center-line": 3.432861017069132, "driven_lanedir_consec": 12.20064691564406, "sim_compute_sim_state": 0.038915868603518165, "sim_compute_performance-ego0": 0.00188107812136635, "sim_compute_performance-ego1": 0.0017921068984007062, "sim_compute_performance-ego2": 0.001755967922353625, "sim_compute_performance-ego3": 0.001773173961909387}, "LFV_multi-norm-techtrack-000-ego3": {"driven_any": 14.590071835060844, "get_ui_image": 0.04324476427083011, "step_physics": 0.4586728026130416, "survival_time": 59.99999999999873, "driven_lanedir": 12.375144087121852, "get_state_dump": 0.008750381715887294, "get_robot_state": 0.013381021802967337, "sim_render-ego0": 0.0035744221581706, "sim_render-ego1": 0.0035646535474791515, "sim_render-ego2": 0.003510539279591531, "sim_render-ego3": 0.003519218430531015, "get_duckie_state": 1.274080300311264e-06, "in-drivable-lane": 5.89999999999991, "deviation-heading": 14.171901553803096, "agent_compute-ego0": 0.042440779103923104, "agent_compute-ego1": 0.0424723609301768, "agent_compute-ego2": 0.04153643301583448, "agent_compute-ego3": 0.0422902015921079, "complete-iteration": 0.7613462413181175, "set_robot_commands": 0.002008956834537401, "deviation-center-line": 3.216440533212472, "driven_lanedir_consec": 12.375144087121852, "sim_compute_sim_state": 0.038915868603518165, "sim_compute_performance-ego0": 0.00188107812136635, "sim_compute_performance-ego1": 0.0017921068984007062, "sim_compute_performance-ego2": 0.001755967922353625, "sim_compute_performance-ego3": 0.001773173961909387}, "LFV_multi-norm-small_loop-000-ego0": {"driven_any": 7.840462106424704, "get_ui_image": 0.029869042190850948, "step_physics": 0.1905649787160606, "survival_time": 33.10000000000026, "driven_lanedir": 7.72351436571707, "get_state_dump": 0.006308674272908345, "get_robot_state": 0.007254326145753242, "sim_render-ego0": 0.003859815554381495, "sim_render-ego1": 0.0038219469165370473, "get_duckie_state": 1.3240681874086774e-06, "in-drivable-lane": 0.0, "deviation-heading": 4.782133849573704, "agent_compute-ego0": 0.04150709739098182, "agent_compute-ego1": 0.042300931289005424, "complete-iteration": 0.34394260697055545, "set_robot_commands": 0.0022614347449255207, "deviation-center-line": 1.4555344056960988, "driven_lanedir_consec": 7.72351436571707, "sim_compute_sim_state": 0.0098266292301598, "sim_compute_performance-ego0": 0.0020261627156989786, "sim_compute_performance-ego1": 0.0020006278341532114}, "LFV_multi-norm-small_loop-000-ego1": {"driven_any": 7.840222418762812, "get_ui_image": 0.029869042190850948, "step_physics": 0.1905649787160606, "survival_time": 33.10000000000026, "driven_lanedir": 5.331399939080234, "get_state_dump": 0.006308674272908345, "get_robot_state": 0.007254326145753242, "sim_render-ego0": 0.003859815554381495, "sim_render-ego1": 0.0038219469165370473, "get_duckie_state": 1.3240681874086774e-06, "in-drivable-lane": 9.100000000000026, "deviation-heading": 8.389907160129924, "agent_compute-ego0": 0.04150709739098182, "agent_compute-ego1": 0.042300931289005424, "complete-iteration": 0.34394260697055545, "set_robot_commands": 0.0022614347449255207, "deviation-center-line": 1.9742263875296313, "driven_lanedir_consec": 5.331399939080234, "sim_compute_sim_state": 0.0098266292301598, "sim_compute_performance-ego0": 0.0020261627156989786, "sim_compute_performance-ego1": 0.0020006278341532114}}
set_robot_commands_max0.0022761166632712425
set_robot_commands_mean0.0021692929997699805
set_robot_commands_median0.0021767346289235288
set_robot_commands_min0.002008956834537401
sim_compute_performance-ego0_max0.002087103353964316
sim_compute_performance-ego0_mean0.002007326909169773
sim_compute_performance-ego0_median0.002044381348914052
sim_compute_performance-ego0_min0.00188107812136635
sim_compute_performance-ego1_max0.002017368067492236
sim_compute_performance-ego1_mean0.0019458062454479003
sim_compute_performance-ego1_median0.0020005329760981044
sim_compute_performance-ego1_min0.0017921068984007062
sim_compute_sim_state_max0.038915868603518165
sim_compute_sim_state_mean0.026783292119810377
sim_compute_sim_state_median0.026022616075933425
sim_compute_sim_state_min0.0098266292301598
sim_render-ego0_max0.003997539614771938
sim_render-ego0_mean0.0038381110127810943
sim_render-ego0_median0.003931518994600549
sim_render-ego0_min0.0035744221581706
sim_render-ego1_max0.004061796643712499
sim_render-ego1_mean0.003821462423254544
sim_render-ego1_median0.003837694831930727
sim_render-ego1_min0.0035646535474791515
simulation-passed1
step_physics_max0.904519536116042
step_physics_mean0.5604784813278922
step_physics_median0.5031998565605095
step_physics_min0.1905649787160606
survival_time_max59.99999999999873
survival_time_mean45.34285714285651
survival_time_min22.15000000000018
No reset possible
5643511120Melisande Tengexercise_ros_templateaido5-LF-sim-validationLFv-simfailedyesnogpu-prod-030:02:39
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 190, in main
    raise InvalidSubmission(msg)
duckietown_challenges.exceptions.InvalidSubmission: Timeout during connection to ego0: <SignalTimeout in state: 2>
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5641911278Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:46
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 48, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 48, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 48, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 48, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5639111278Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:41
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 48, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 48, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 48, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 48, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5636311430Frank (Chude) QianΒ πŸ‡¨πŸ‡¦template-pytorchaido5-LFV-sim-validationLFVv-simhost-erroryesnogpu-prod-030:00:56
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5626211133Mo KleitΒ πŸ‡¨πŸ‡¦real-exercise-1aido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:05:50
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5619911280Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:03:44
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5616811310Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:02:20
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5615411281Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:52
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5577911420Frank (Chude) QianΒ πŸ‡¨πŸ‡¦challenge-aido_LF-baseline-behavior-cloningaido5-LFVI-sim-validationLFVIv-simsuccessyesnogpu-prod-030:33:49
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median7.695389663561643
survival_time_median33.64999999999935
deviation-center-line_median1.6030631249677336
in-drivable-lane_median0.6999999999999975


other stats
agent_compute-ego0_max0.0459705501997552
agent_compute-ego0_mean0.043270612593259525
agent_compute-ego0_median0.043270612593259525
agent_compute-ego0_min0.04057067498676386
agent_compute-npc0_max0.03806738194379084
agent_compute-npc0_mean0.03643575925923233
agent_compute-npc0_median0.03643575925923233
agent_compute-npc0_min0.034804136574673815
agent_compute-npc1_max0.03894900799194641
agent_compute-npc1_mean0.036804728007193296
agent_compute-npc1_median0.036804728007193296
agent_compute-npc1_min0.03466044802244018
agent_compute-npc2_max0.040291824904607795
agent_compute-npc2_mean0.037794634491091945
agent_compute-npc2_median0.037794634491091945
agent_compute-npc2_min0.035297444077576096
complete-iteration_max1.1698413495303823
complete-iteration_mean1.0594548640394603
complete-iteration_median1.0594548640394603
complete-iteration_min0.9490683785485388
deviation-center-line_max2.8305007562607853
deviation-center-line_mean1.6030631249677336
deviation-center-line_min0.3756254936746817
deviation-heading_max10.829028225443134
deviation-heading_mean6.415621025559583
deviation-heading_median6.415621025559583
deviation-heading_min2.002213825676032
driven_any_max14.590269734375006
driven_any_mean8.0605820989866
driven_any_median8.0605820989866
driven_any_min1.530894463598194
driven_lanedir_consec_max14.2852209595819
driven_lanedir_consec_mean7.695389663561643
driven_lanedir_consec_min1.105558367541386
driven_lanedir_max14.285221064869503
driven_lanedir_mean7.695676408988919
driven_lanedir_median7.695676408988919
driven_lanedir_min1.1061317531083337
get_duckie_state_max1.2391278532897535e-06
get_duckie_state_mean1.2223599428329655e-06
get_duckie_state_median1.2223599428329655e-06
get_duckie_state_min1.2055920323761771e-06
get_robot_state_max0.013613084546562768
get_robot_state_mean0.013351758490608264
get_robot_state_median0.013351758490608264
get_robot_state_min0.01309043243465376
get_state_dump_max0.008871112551007952
get_state_dump_mean0.008862859062452554
get_state_dump_median0.008862859062452554
get_state_dump_min0.008854605573897159
get_ui_image_max0.047108959989482856
get_ui_image_mean0.046735399518083795
get_ui_image_median0.046735399518083795
get_ui_image_min0.04636183904668473
in-drivable-lane_max1.399999999999995
in-drivable-lane_mean0.6999999999999975
in-drivable-lane_min0.0
per-episodes
details{"LFVI-norm-4way-000-ego0": {"driven_any": 14.590269734375006, "get_ui_image": 0.04636183904668473, "step_physics": 0.6558148368609934, "survival_time": 59.99999999999873, "driven_lanedir": 14.285221064869503, "get_state_dump": 0.008854605573897159, "get_robot_state": 0.01309043243465376, "sim_render-ego0": 0.0035800991407739036, "sim_render-npc0": 0.003563268893366551, "sim_render-npc1": 0.0035743498980850105, "sim_render-npc2": 0.003602111468604959, "get_duckie_state": 1.2055920323761771e-06, "in-drivable-lane": 0.0, "deviation-heading": 10.829028225443134, "agent_compute-ego0": 0.04057067498676386, "agent_compute-npc0": 0.03806738194379084, "agent_compute-npc1": 0.03894900799194641, "agent_compute-npc2": 0.040291824904607795, "complete-iteration": 0.9490683785485388, "set_robot_commands": 0.0020528574172503544, "deviation-center-line": 2.8305007562607853, "driven_lanedir_consec": 14.2852209595819, "sim_compute_sim_state": 0.03713664682977504, "sim_compute_performance-ego0": 0.0018550871214600623, "sim_compute_performance-npc0": 0.0017834126601906045, "sim_compute_performance-npc1": 0.001821917955524022, "sim_compute_performance-npc2": 0.001802483565801387}, "LFVI-norm-udem1-000-ego0": {"driven_any": 1.530894463598194, "get_ui_image": 0.047108959989482856, "step_physics": 0.8803340350689531, "survival_time": 7.299999999999982, "driven_lanedir": 1.1061317531083337, "get_state_dump": 0.008871112551007952, "get_robot_state": 0.013613084546562768, "sim_render-ego0": 0.0036531029915323064, "sim_render-npc0": 0.0038044095850315223, "sim_render-npc1": 0.003716227148665863, "sim_render-npc2": 0.003641798382713681, "get_duckie_state": 1.2391278532897535e-06, "in-drivable-lane": 1.399999999999995, "deviation-heading": 2.002213825676032, "agent_compute-ego0": 0.0459705501997552, "agent_compute-npc0": 0.034804136574673815, "agent_compute-npc1": 0.03466044802244018, "agent_compute-npc2": 0.035297444077576096, "complete-iteration": 1.1698413495303823, "set_robot_commands": 0.0021905623325685253, "deviation-center-line": 0.3756254936746817, "driven_lanedir_consec": 1.105558367541386, "sim_compute_sim_state": 0.03796529283328933, "sim_compute_performance-ego0": 0.001921717001467335, "sim_compute_performance-npc0": 0.001917741736587213, "sim_compute_performance-npc1": 0.0018273437915205144, "sim_compute_performance-npc2": 0.0018274573241772295}}
set_robot_commands_max0.0021905623325685253
set_robot_commands_mean0.00212170987490944
set_robot_commands_median0.00212170987490944
set_robot_commands_min0.0020528574172503544
sim_compute_performance-ego0_max0.001921717001467335
sim_compute_performance-ego0_mean0.001888402061463699
sim_compute_performance-ego0_median0.001888402061463699
sim_compute_performance-ego0_min0.0018550871214600623
sim_compute_performance-npc0_max0.001917741736587213
sim_compute_performance-npc0_mean0.0018505771983889088
sim_compute_performance-npc0_median0.0018505771983889088
sim_compute_performance-npc0_min0.0017834126601906045
sim_compute_performance-npc1_max0.0018273437915205144
sim_compute_performance-npc1_mean0.001824630873522268
sim_compute_performance-npc1_median0.001824630873522268
sim_compute_performance-npc1_min0.001821917955524022
sim_compute_performance-npc2_max0.0018274573241772295
sim_compute_performance-npc2_mean0.0018149704449893084
sim_compute_performance-npc2_median0.0018149704449893084
sim_compute_performance-npc2_min0.001802483565801387
sim_compute_sim_state_max0.03796529283328933
sim_compute_sim_state_mean0.03755096983153218
sim_compute_sim_state_median0.03755096983153218
sim_compute_sim_state_min0.03713664682977504
sim_render-ego0_max0.0036531029915323064
sim_render-ego0_mean0.0036166010661531057
sim_render-ego0_median0.0036166010661531057
sim_render-ego0_min0.0035800991407739036
sim_render-npc0_max0.0038044095850315223
sim_render-npc0_mean0.003683839239199037
sim_render-npc0_median0.003683839239199037
sim_render-npc0_min0.003563268893366551
sim_render-npc1_max0.003716227148665863
sim_render-npc1_mean0.0036452885233754367
sim_render-npc1_median0.0036452885233754367
sim_render-npc1_min0.0035743498980850105
sim_render-npc2_max0.003641798382713681
sim_render-npc2_mean0.00362195492565932
sim_render-npc2_median0.00362195492565932
sim_render-npc2_min0.003602111468604959
simulation-passed1
step_physics_max0.8803340350689531
step_physics_mean0.7680744359649733
step_physics_median0.7680744359649733
step_physics_min0.6558148368609934
survival_time_max59.99999999999873
survival_time_mean33.64999999999935
survival_time_min7.299999999999982
No reset possible
5575311282Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:37
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5566911285Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:02:07
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5563311287Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:01:28
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5558411284Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:02:06
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5555711299Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:46
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5551811286Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:33
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5547411301Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:42
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5538911283Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:02:16
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5536711290Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:42
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5532011291Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:02:19
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5529811296Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:38
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5528011296Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:40
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5525711289Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:43
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5521811309Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:47
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5520411303Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:44
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5516811450Cliff ChandlerΒ πŸ‡ΊπŸ‡Έtemplate-pytorchaido5-LF-sim-testingLFt-simhost-erroryesnogpu-prod-030:02:18
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5514211308Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:43
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5512211302Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:47
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5511511302Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:38
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5507411306Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:40
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5505411306Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:59
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5501811413Frank (Chude) QianΒ πŸ‡¨πŸ‡¦challenge-aido_LF-baseline-behavior-cloningaido5-LFP-sim-validationLFP-simhost-errornonogpu-prod-030:03:15
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 53, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 777, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 991, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 120, in write_logs
    services2id: Dict[ServiceName, ContainerID] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 63, in get_services_id
    raise DockerComposeFail(msg, output=output.decode(), names=names) from e
duckietown_challenges_runner.docker_compose.DockerComposeFail: Cannot get process ids
β”‚ output: ''
β”‚  names: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5498611305Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:36
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5496611307Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:33
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5492811311Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:02:56
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5492011436Charlie GauthierΒ πŸ‡¨πŸ‡¦baseline-RL-sim-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:42
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 23, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 43, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 23, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 43, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 23, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 43, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 23, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 43, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5479211366Yishu MalhotraΒ πŸ‡¨πŸ‡¦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-030:20:22
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.878808220803569
survival_time_median32.47500000000015
deviation-center-line_median0.8895903227463957
in-drivable-lane_median10.900000000000114


other stats
agent_compute-ego0_max0.012722690704420867
agent_compute-ego0_mean0.01207844123806592
agent_compute-ego0_median0.011937871154352284
agent_compute-ego0_min0.011715331939138242
complete-iteration_max0.1981699437009058
complete-iteration_mean0.17873993354482867
complete-iteration_median0.18279336512442787
complete-iteration_min0.15120306022955318
deviation-center-line_max1.715956032690052
deviation-center-line_mean0.9277220301444664
deviation-center-line_min0.21575144239502256
deviation-heading_max5.975266666435523
deviation-heading_mean3.6506651793318183
deviation-heading_median3.963888066486729
deviation-heading_min0.6996179179182931
driven_any_max10.91785339974512
driven_any_mean6.330381778383487
driven_any_median5.612037995914645
driven_any_min3.1795977219595355
driven_lanedir_consec_max5.935940325394702
driven_lanedir_consec_mean2.596439254251906
driven_lanedir_consec_min0.6922002500057847
driven_lanedir_max5.935940325394702
driven_lanedir_mean2.667843000783218
driven_lanedir_median2.0216157138661925
driven_lanedir_min0.6922002500057847
get_duckie_state_max1.2453257311664835e-06
get_duckie_state_mean1.2120185669110667e-06
get_duckie_state_median1.2202165743839172e-06
get_duckie_state_min1.1623153877099488e-06
get_robot_state_max0.0037295025557720018
get_robot_state_mean0.003653786781760586
get_robot_state_median0.003669817857003514
get_robot_state_min0.0035460088572633157
get_state_dump_max0.00470602629807195
get_state_dump_mean0.004629751212943744
get_state_dump_median0.004628323523116121
get_state_dump_min0.004556331507470784
get_ui_image_max0.03440786680717061
get_ui_image_mean0.02990009772198879
get_ui_image_median0.03011718655616013
get_ui_image_min0.02495815096846429
in-drivable-lane_max56.249999999998735
in-drivable-lane_mean20.937499999999673
in-drivable-lane_min5.699999999999733
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 6.710110986124219, "get_ui_image": 0.02825865984285636, "step_physics": 0.10790835310867578, "survival_time": 36.90000000000004, "driven_lanedir": 5.935940325394702, "get_state_dump": 0.004647224294639246, "get_robot_state": 0.003706912065229816, "sim_render-ego0": 0.0038116688012108913, "get_duckie_state": 1.2453257311664835e-06, "in-drivable-lane": 5.699999999999733, "deviation-heading": 4.5846260843909326, "agent_compute-ego0": 0.012001584121435678, "complete-iteration": 0.17456154184509196, "set_robot_commands": 0.0022168069150354285, "deviation-center-line": 1.715956032690052, "driven_lanedir_consec": 5.935940325394702, "sim_compute_sim_state": 0.009942463834811613, "sim_compute_performance-ego0": 0.0019769142703209906}, "LF-norm-zigzag-000-ego0": {"driven_any": 4.513965005705071, "get_ui_image": 0.03440786680717061, "step_physics": 0.12507663714928136, "survival_time": 28.050000000000264, "driven_lanedir": 2.68683244512118, "get_state_dump": 0.0046094227515929955, "get_robot_state": 0.003632723648777212, "sim_render-ego0": 0.003762214633493661, "get_duckie_state": 1.2031229365338634e-06, "in-drivable-lane": 10.100000000000144, "deviation-heading": 5.975266666435523, "agent_compute-ego0": 0.011874158187268892, "complete-iteration": 0.1981699437009058, "set_robot_commands": 0.0021637245429368206, "deviation-center-line": 1.206148601246852, "driven_lanedir_consec": 2.672672539880646, "sim_compute_sim_state": 0.010595181658598875, "sim_compute_performance-ego0": 0.001959665814328448}, "LF-norm-techtrack-000-ego0": {"driven_any": 3.1795977219595355, "get_ui_image": 0.0319757132694639, "step_physics": 0.11889110175259594, "survival_time": 20.250000000000153, "driven_lanedir": 1.3563989826112044, "get_state_dump": 0.00470602629807195, "get_robot_state": 0.0037295025557720018, "sim_render-ego0": 0.003862969981038512, "get_duckie_state": 1.2373102122339708e-06, "in-drivable-lane": 11.700000000000085, "deviation-heading": 3.343150048582525, "agent_compute-ego0": 0.012722690704420867, "complete-iteration": 0.19102518840376376, "set_robot_commands": 0.0022938656689498226, "deviation-center-line": 0.5730320442459397, "driven_lanedir_consec": 1.084943901726492, "sim_compute_sim_state": 0.0107068604436414, "sim_compute_performance-ego0": 0.0020415495181905813}, "LF-norm-small_loop-000-ego0": {"driven_any": 10.91785339974512, "get_ui_image": 0.02495815096846429, "step_physics": 0.0928324823276288, "survival_time": 59.99999999999873, "driven_lanedir": 0.6922002500057847, "get_state_dump": 0.004556331507470784, "get_robot_state": 0.0035460088572633157, "sim_render-ego0": 0.003659371630933064, "get_duckie_state": 1.1623153877099488e-06, "in-drivable-lane": 56.249999999998735, "deviation-heading": 0.6996179179182931, "agent_compute-ego0": 0.011715331939138242, "complete-iteration": 0.15120306022955318, "set_robot_commands": 0.0021156591737796424, "deviation-center-line": 0.21575144239502256, "driven_lanedir_consec": 0.6922002500057847, "sim_compute_sim_state": 0.005870567769631061, "sim_compute_performance-ego0": 0.001863812328278274}}
set_robot_commands_max0.0022938656689498226
set_robot_commands_mean0.0021975140751754286
set_robot_commands_median0.0021902657289861243
set_robot_commands_min0.0021156591737796424
sim_compute_performance-ego0_max0.0020415495181905813
sim_compute_performance-ego0_mean0.001960485482779574
sim_compute_performance-ego0_median0.0019682900423247194
sim_compute_performance-ego0_min0.001863812328278274
sim_compute_sim_state_max0.0107068604436414
sim_compute_sim_state_mean0.009278768426670738
sim_compute_sim_state_median0.010268822746705243
sim_compute_sim_state_min0.005870567769631061
sim_render-ego0_max0.003862969981038512
sim_render-ego0_mean0.003774056261669032
sim_render-ego0_median0.003786941717352276
sim_render-ego0_min0.003659371630933064
simulation-passed1
step_physics_max0.12507663714928136
step_physics_mean0.11117714358454547
step_physics_median0.11339972743063588
step_physics_min0.0928324823276288
survival_time_max59.99999999999873
survival_time_mean36.2999999999998
survival_time_min20.250000000000153
No reset possible
5474911376Charlie GauthierΒ πŸ‡¨πŸ‡¦template-rpl-rosaido5-LF-sim-validationLFv-simfailedyesnogpu-prod-030:03:35
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 190, in main
    raise InvalidSubmission(msg)
duckietown_challenges.exceptions.InvalidSubmission: Timeout during connection to ego0: <SignalTimeout in state: 2>
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5473711439Charlie GauthierΒ πŸ‡¨πŸ‡¦baseline-RL-sim-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:01:08
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 23, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 43, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 23, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 43, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 23, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 43, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 23, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 43, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5468911398Charlie GauthierΒ πŸ‡¨πŸ‡¦baseline-RL-sim-pytorchaido5-LF-sim-validationLFv-simfailedyesnogpu-prod-030:05:26
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 190, in main
    raise InvalidSubmission(msg)
duckietown_challenges.exceptions.InvalidSubmission: Timeout during connection to ego0: <SignalTimeout in state: 2>
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5466511424Frank (Chude) QianΒ πŸ‡¨πŸ‡¦template-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:45
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5466111398Charlie GauthierΒ πŸ‡¨πŸ‡¦baseline-RL-sim-pytorchaido5-LF-sim-validationLFv-simfailedyesnogpu-prod-030:00:45
The container "solut [...]
The container "solution-ego0" exited with code 127.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5449711485Raphael Jeanreal-exercise-2aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-030:40:59
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median8.058347075976958
survival_time_median59.99999999999873
deviation-center-line_median3.941272959315912
in-drivable-lane_median3.7499999999998983


other stats
agent_compute-ego0_max0.013271235109467393
agent_compute-ego0_mean0.01274713747308812
agent_compute-ego0_median0.012740427607997668
agent_compute-ego0_min0.012236459566889754
complete-iteration_max0.28653315937985585
complete-iteration_mean0.24894970992919707
complete-iteration_median0.2514519946561269
complete-iteration_min0.20636169102467863
deviation-center-line_max4.227889538881097
deviation-center-line_mean3.958511662188766
deviation-center-line_min3.723611191242143
deviation-heading_max24.968836989331013
deviation-heading_mean24.15339382114701
deviation-heading_median24.51043192659232
deviation-heading_min22.623874442072392
driven_any_max11.604236830567826
driven_any_mean11.28549893698308
driven_any_median11.228234839116054
driven_any_min11.081289239132389
driven_lanedir_consec_max9.693542784705771
driven_lanedir_consec_mean7.636637124154132
driven_lanedir_consec_min4.736311559956839
driven_lanedir_max9.693542784705771
driven_lanedir_mean9.340914719039512
driven_lanedir_median9.246840937956003
driven_lanedir_min9.176434215540263
get_duckie_state_max1.585553031876919e-06
get_duckie_state_mean1.3628669027285611e-06
get_duckie_state_median1.3218235711471722e-06
get_duckie_state_min1.222267436742981e-06
get_robot_state_max0.003928737179822072
get_robot_state_mean0.003819166472512022
get_robot_state_median0.00382316787475154
get_robot_state_min0.0037015929607229366
get_state_dump_max0.005006426081470804
get_state_dump_mean0.004846540219976344
get_state_dump_median0.004867399264930388
get_state_dump_min0.004644936268573796
get_ui_image_max0.03634925567538018
get_ui_image_mean0.031051417870088777
get_ui_image_median0.030703166442350185
get_ui_image_min0.026450082920274568
in-drivable-lane_max4.899999999999775
in-drivable-lane_mean3.3999999999998884
in-drivable-lane_min1.1999999999999815
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 11.20852766652064, "get_ui_image": 0.02872768309987852, "step_physics": 0.1654628551969123, "survival_time": 59.99999999999873, "driven_lanedir": 9.176434215540263, "get_state_dump": 0.005006426081470804, "get_robot_state": 0.003928737179822072, "sim_render-ego0": 0.003966054749627792, "get_duckie_state": 1.585553031876919e-06, "in-drivable-lane": 4.899999999999775, "deviation-heading": 22.623874442072392, "agent_compute-ego0": 0.012912891191010868, "complete-iteration": 0.2341311382909897, "set_robot_commands": 0.002375438350324925, "deviation-center-line": 3.844704008080269, "driven_lanedir_consec": 9.176434215540263, "sim_compute_sim_state": 0.0094782388974586, "sim_compute_performance-ego0": 0.002168596237525654}, "LF-norm-zigzag-000-ego0": {"driven_any": 11.081289239132389, "get_ui_image": 0.03634925567538018, "step_physics": 0.21014007720026148, "survival_time": 59.99999999999873, "driven_lanedir": 9.244684737643292, "get_state_dump": 0.004644936268573796, "get_robot_state": 0.0037015929607229366, "sim_render-ego0": 0.003818323967558061, "get_duckie_state": 1.2300095887704255e-06, "in-drivable-lane": 2.6999999999999567, "deviation-heading": 24.968836989331013, "agent_compute-ego0": 0.012236459566889754, "complete-iteration": 0.28653315937985585, "set_robot_commands": 0.002248969502889743, "deviation-center-line": 4.037841910551555, "driven_lanedir_consec": 4.736311559956839, "sim_compute_sim_state": 0.011259096051929991, "sim_compute_performance-ego0": 0.0020466212130506073}, "LF-norm-techtrack-000-ego0": {"driven_any": 11.247942011711467, "get_ui_image": 0.03267864978482185, "step_physics": 0.19200843756244543, "survival_time": 59.99999999999873, "driven_lanedir": 9.693542784705771, "get_state_dump": 0.004997366969531819, "get_robot_state": 0.0039134845447778505, "sim_render-ego0": 0.0039756409234547995, "get_duckie_state": 1.413637553523919e-06, "in-drivable-lane": 1.1999999999999815, "deviation-heading": 24.16885500116777, "agent_compute-ego0": 0.013271235109467393, "complete-iteration": 0.268772851021264, "set_robot_commands": 0.002335052704632431, "deviation-center-line": 3.723611191242143, "driven_lanedir_consec": 9.693542784705771, "sim_compute_sim_state": 0.013283956457832076, "sim_compute_performance-ego0": 0.002203926853494382}, "LF-norm-small_loop-000-ego0": {"driven_any": 11.604236830567826, "get_ui_image": 0.026450082920274568, "step_physics": 0.14436405207294906, "survival_time": 59.99999999999873, "driven_lanedir": 9.248997138268718, "get_state_dump": 0.004737431560328958, "get_robot_state": 0.003732851204725229, "sim_render-ego0": 0.0038023864498344887, "get_duckie_state": 1.222267436742981e-06, "in-drivable-lane": 4.799999999999841, "deviation-heading": 24.852008852016866, "agent_compute-ego0": 0.012567964024984469, "complete-iteration": 0.20636169102467863, "set_robot_commands": 0.0023224318057273844, "deviation-center-line": 4.227889538881097, "driven_lanedir_consec": 6.940259936413649, "sim_compute_sim_state": 0.006261917871797611, "sim_compute_performance-ego0": 0.0020336548950551053}}
set_robot_commands_max0.002375438350324925
set_robot_commands_mean0.0023204730908936207
set_robot_commands_median0.0023287422551799076
set_robot_commands_min0.002248969502889743
sim_compute_performance-ego0_max0.002203926853494382
sim_compute_performance-ego0_mean0.0021131997997814373
sim_compute_performance-ego0_median0.0021076087252881307
sim_compute_performance-ego0_min0.0020336548950551053
sim_compute_sim_state_max0.013283956457832076
sim_compute_sim_state_mean0.01007080231975457
sim_compute_sim_state_median0.010368667474694296
sim_compute_sim_state_min0.006261917871797611
sim_render-ego0_max0.0039756409234547995
sim_render-ego0_mean0.0038906015226187855
sim_render-ego0_median0.003892189358592927
sim_render-ego0_min0.0038023864498344887
simulation-passed1
step_physics_max0.21014007720026148
step_physics_mean0.17799385550814206
step_physics_median0.17873564637967887
step_physics_min0.14436405207294906
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
5443711543Yishu MalhotraΒ πŸ‡¨πŸ‡¦sim-exercise-2aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-030:26:26
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median6.229995277573435
survival_time_median44.1999999999995
deviation-center-line_median2.182068153318206
in-drivable-lane_median3.725000000000053


other stats
agent_compute-ego0_max0.013610877973962123
agent_compute-ego0_mean0.013175286533413242
agent_compute-ego0_median0.01311502738479571
agent_compute-ego0_min0.012860213390099417
complete-iteration_max0.2283041068258929
complete-iteration_mean0.19695918121253717
complete-iteration_median0.1927205133334545
complete-iteration_min0.17409159135734678
deviation-center-line_max4.398214432037851
deviation-center-line_mean2.258744777995552
deviation-center-line_min0.2726283733079452
deviation-heading_max14.300149212732972
deviation-heading_mean7.780347288218778
deviation-heading_median8.063353571567086
deviation-heading_min0.6945327970079658
driven_any_max12.03990369032132
driven_any_mean7.92673626622937
driven_any_median7.990752704263581
driven_any_min3.685535966069
driven_lanedir_consec_max11.539878592945154
driven_lanedir_consec_mean6.186246469242104
driven_lanedir_consec_min0.7451167288763929
driven_lanedir_max11.539878592945154
driven_lanedir_mean6.186246469242104
driven_lanedir_median6.229995277573435
driven_lanedir_min0.7451167288763929
get_duckie_state_max1.5017437390158172e-06
get_duckie_state_mean1.4399841900348497e-06
get_duckie_state_median1.476269578258759e-06
get_duckie_state_min1.305653864606063e-06
get_robot_state_max0.00385250902762522
get_robot_state_mean0.003822895989749199
get_robot_state_median0.003820011756700044
get_robot_state_min0.00379905141797149
get_state_dump_max0.00568071386760607
get_state_dump_mean0.005045706571826384
get_state_dump_median0.004877562529300727
get_state_dump_min0.004746987361098011
get_ui_image_max0.036630949509530934
get_ui_image_mean0.031219869181224477
get_ui_image_median0.0306170864022709
get_ui_image_min0.02701435441082517
in-drivable-lane_max24.55000000000027
in-drivable-lane_mean8.000000000000096
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 12.03990369032132, "get_ui_image": 0.02914718684308435, "step_physics": 0.11828811182566824, "survival_time": 59.99999999999873, "driven_lanedir": 11.539878592945154, "get_state_dump": 0.004746987361098011, "get_robot_state": 0.0038294949003500702, "sim_render-ego0": 0.003906328017070431, "get_duckie_state": 1.4805376877097862e-06, "in-drivable-lane": 0.25000000000000355, "deviation-heading": 13.370134023692817, "agent_compute-ego0": 0.012860213390099417, "complete-iteration": 0.18687098210896183, "set_robot_commands": 0.0023350781147724185, "deviation-center-line": 3.5384731247714183, "driven_lanedir_consec": 11.539878592945154, "sim_compute_sim_state": 0.009553401694508218, "sim_compute_performance-ego0": 0.002114846049300042}, "LF-norm-zigzag-000-ego0": {"driven_any": 10.625171265047664, "get_ui_image": 0.036630949509530934, "step_physics": 0.1482141006002815, "survival_time": 59.99999999999873, "driven_lanedir": 10.168415679697215, "get_state_dump": 0.00476457375868671, "get_robot_state": 0.0038105286130500175, "sim_render-ego0": 0.003957769058824678, "get_duckie_state": 1.4720014688077318e-06, "in-drivable-lane": 0.0, "deviation-heading": 14.300149212732972, "agent_compute-ego0": 0.012908558762143, "complete-iteration": 0.2283041068258929, "set_robot_commands": 0.0023317364828473425, "deviation-center-line": 4.398214432037851, "driven_lanedir_consec": 10.168415679697215, "sim_compute_sim_state": 0.013468949423543024, "sim_compute_performance-ego0": 0.002125104996286562}, "LF-norm-techtrack-000-ego0": {"driven_any": 3.685535966069, "get_ui_image": 0.03208698596145744, "step_physics": 0.12460312819540352, "survival_time": 20.00000000000015, "driven_lanedir": 2.291574875449653, "get_state_dump": 0.00568071386760607, "get_robot_state": 0.00379905141797149, "sim_render-ego0": 0.003796828952513431, "get_duckie_state": 1.305653864606063e-06, "in-drivable-lane": 7.200000000000102, "deviation-heading": 2.7565731194413563, "agent_compute-ego0": 0.01332149600744842, "complete-iteration": 0.19857004455794716, "set_robot_commands": 0.0023632293330166404, "deviation-center-line": 0.8256631818649931, "driven_lanedir_consec": 2.291574875449653, "sim_compute_sim_state": 0.0108395092505172, "sim_compute_performance-ego0": 0.0019930645711998688}, "LF-norm-small_loop-000-ego0": {"driven_any": 5.356334143479498, "get_ui_image": 0.02701435441082517, "step_physics": 0.10942441340280544, "survival_time": 28.40000000000027, "driven_lanedir": 0.7451167288763929, "get_state_dump": 0.004990551299914744, "get_robot_state": 0.00385250902762522, "sim_render-ego0": 0.003974992487258777, "get_duckie_state": 1.5017437390158172e-06, "in-drivable-lane": 24.55000000000027, "deviation-heading": 0.6945327970079658, "agent_compute-ego0": 0.013610877973962123, "complete-iteration": 0.17409159135734678, "set_robot_commands": 0.0023378278333399125, "deviation-center-line": 0.2726283733079452, "driven_lanedir_consec": 0.7451167288763929, "sim_compute_sim_state": 0.006745796840513528, "sim_compute_performance-ego0": 0.0020502535446247443}}
set_robot_commands_max0.0023632293330166404
set_robot_commands_mean0.002341967940994078
set_robot_commands_median0.0023364529740561657
set_robot_commands_min0.0023317364828473425
sim_compute_performance-ego0_max0.002125104996286562
sim_compute_performance-ego0_mean0.0020708172903528043
sim_compute_performance-ego0_median0.002082549796962393
sim_compute_performance-ego0_min0.0019930645711998688
sim_compute_sim_state_max0.013468949423543024
sim_compute_sim_state_mean0.010151914302270491
sim_compute_sim_state_median0.01019645547251271
sim_compute_sim_state_min0.006745796840513528
sim_render-ego0_max0.003974992487258777
sim_render-ego0_mean0.003908979628916829
sim_render-ego0_median0.003932048537947554
sim_render-ego0_min0.003796828952513431
simulation-passed1
step_physics_max0.1482141006002815
step_physics_mean0.12513243850603967
step_physics_median0.12144562001053588
step_physics_min0.10942441340280544
survival_time_max59.99999999999873
survival_time_mean42.09999999999947
survival_time_min20.00000000000015
No reset possible
5439111557Olga (Ge Ya) XuΒ πŸ‡¨πŸ‡¦sim-exercise-2aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-030:15:53
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.9303925832164944
survival_time_median19.30000000000014
deviation-center-line_median0.6361587648158792
in-drivable-lane_median12.125000000000083


other stats
agent_compute-ego0_max0.015526942908763884
agent_compute-ego0_mean0.01301255770686021
agent_compute-ego0_median0.012676055645900682
agent_compute-ego0_min0.011171176626875596
complete-iteration_max0.2221273792826611
complete-iteration_mean0.19084872941429892
complete-iteration_median0.19390960130762996
complete-iteration_min0.1534483357592746
deviation-center-line_max3.0517027099096983
deviation-center-line_mean1.1046066467253002
deviation-center-line_min0.09440634735974372
deviation-heading_max9.478236434594477
deviation-heading_mean4.266769740416985
deviation-heading_median3.554602297598159
deviation-heading_min0.47963793187714326
driven_any_max12.490709784444984
driven_any_mean5.057792979368167
driven_any_median3.3373235724872123
driven_any_min1.0658149880532597
driven_lanedir_consec_max7.238859430628122
driven_lanedir_consec_mean2.348267392403378
driven_lanedir_consec_min0.2934249725524003
driven_lanedir_max7.238859430628122
driven_lanedir_mean2.539056099416861
driven_lanedir_median1.3119699972434604
driven_lanedir_min0.2934249725524003
get_duckie_state_max1.539786656697591e-06
get_duckie_state_mean1.3358556287905388e-06
get_duckie_state_median1.3149001207683007e-06
get_duckie_state_min1.173835616927963e-06
get_robot_state_max0.003998639682928721
get_robot_state_mean0.0037972378820191745
get_robot_state_median0.0038454717777892497
get_robot_state_min0.003499368289569477
get_state_dump_max0.005154331525166829
get_state_dump_mean0.00487239509728746
get_state_dump_median0.004865045454355117
get_state_dump_min0.004605157955272778
get_ui_image_max0.03754204813984857
get_ui_image_mean0.031212747046998016
get_ui_image_median0.03096882851193125
get_ui_image_min0.025371283024280995
in-drivable-lane_max24.949999999999083
in-drivable-lane_mean12.99999999999981
in-drivable-lane_min2.79999999999999
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 12.490709784444984, "get_ui_image": 0.028984635596866908, "step_physics": 0.10312581419646988, "survival_time": 59.99999999999873, "driven_lanedir": 7.238859430628122, "get_state_dump": 0.004742897718177052, "get_robot_state": 0.003758370330391279, "sim_render-ego0": 0.003901706547860202, "get_duckie_state": 1.2748743671858737e-06, "in-drivable-lane": 24.949999999999083, "deviation-heading": 9.478236434594477, "agent_compute-ego0": 0.012436859812168754, "complete-iteration": 0.17110338993215443, "set_robot_commands": 0.0022591045754438237, "deviation-center-line": 3.0517027099096983, "driven_lanedir_consec": 7.238859430628122, "sim_compute_sim_state": 0.00975664747843238, "sim_compute_performance-ego0": 0.0020456028222839203}, "LF-norm-zigzag-000-ego0": {"driven_any": 4.876491830916982, "get_ui_image": 0.03754204813984857, "step_physics": 0.1420548463213271, "survival_time": 27.550000000000257, "driven_lanedir": 1.980036430957537, "get_state_dump": 0.004987193190533182, "get_robot_state": 0.003932573225187219, "sim_render-ego0": 0.004059015408806179, "get_duckie_state": 1.3549258743507275e-06, "in-drivable-lane": 14.850000000000149, "deviation-heading": 6.612885520178503, "agent_compute-ego0": 0.012915251479632612, "complete-iteration": 0.2221273792826611, "set_robot_commands": 0.0024253151554992232, "deviation-center-line": 1.1687403363607405, "driven_lanedir_consec": 1.2168816029036047, "sim_compute_sim_state": 0.011932576048201408, "sim_compute_performance-ego0": 0.002180784076884173}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.0658149880532597, "get_ui_image": 0.0329530214269956, "step_physics": 0.141758494079113, "survival_time": 4.749999999999991, "driven_lanedir": 0.643903563529384, "get_state_dump": 0.005154331525166829, "get_robot_state": 0.003998639682928721, "sim_render-ego0": 0.004077233374118805, "get_duckie_state": 1.539786656697591e-06, "in-drivable-lane": 2.79999999999999, "deviation-heading": 0.47963793187714326, "agent_compute-ego0": 0.015526942908763884, "complete-iteration": 0.21671581268310547, "set_robot_commands": 0.0024019752939542136, "deviation-center-line": 0.09440634735974372, "driven_lanedir_consec": 0.643903563529384, "sim_compute_sim_state": 0.00855878492196401, "sim_compute_performance-ego0": 0.0021873141328493753}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.7981553140574429, "get_ui_image": 0.025371283024280995, "step_physics": 0.09615313684618151, "survival_time": 11.050000000000022, "driven_lanedir": 0.2934249725524003, "get_state_dump": 0.004605157955272778, "get_robot_state": 0.003499368289569477, "sim_render-ego0": 0.0035591877258575716, "get_duckie_state": 1.173835616927963e-06, "in-drivable-lane": 9.40000000000002, "deviation-heading": 0.4963190750178152, "agent_compute-ego0": 0.011171176626875596, "complete-iteration": 0.1534483357592746, "set_robot_commands": 0.0021247767113350535, "deviation-center-line": 0.10357719327101796, "driven_lanedir_consec": 0.2934249725524003, "sim_compute_sim_state": 0.005014661196115855, "sim_compute_performance-ego0": 0.0018664501808785105}}
set_robot_commands_max0.0024253151554992232
set_robot_commands_mean0.0023027929340580784
set_robot_commands_median0.0023305399346990187
set_robot_commands_min0.0021247767113350535
sim_compute_performance-ego0_max0.0021873141328493753
sim_compute_performance-ego0_mean0.0020700378032239945
sim_compute_performance-ego0_median0.0021131934495840464
sim_compute_performance-ego0_min0.0018664501808785105
sim_compute_sim_state_max0.011932576048201408
sim_compute_sim_state_mean0.008815667411178413
sim_compute_sim_state_median0.009157716200198196
sim_compute_sim_state_min0.005014661196115855
sim_render-ego0_max0.004077233374118805
sim_render-ego0_mean0.0038992857641606898
sim_render-ego0_median0.003980360978333191
sim_render-ego0_min0.0035591877258575716
simulation-passed1
step_physics_max0.1420548463213271
step_physics_mean0.12077307286077288
step_physics_median0.12244215413779144
step_physics_min0.09615313684618151
survival_time_max59.99999999999873
survival_time_mean25.83749999999975
survival_time_min4.749999999999991
No reset possible
5428711601Jean-SΓ©bastien GrondinΒ πŸ‡¨πŸ‡¦sim-exercise-2aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-030:28:12
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median2.352416319083252
survival_time_median55.174999999999
deviation-center-line_median1.7338464226629915
in-drivable-lane_median32.02499999999935


other stats
agent_compute-ego0_max0.012830386650155326
agent_compute-ego0_mean0.012589615512786844
agent_compute-ego0_median0.012593538872402813
agent_compute-ego0_min0.012340997656186422
complete-iteration_max0.30018571217854817
complete-iteration_mean0.2458990501133769
complete-iteration_median0.23908016533587123
complete-iteration_min0.205250157603217
deviation-center-line_max2.09239612944691
deviation-center-line_mean1.4302356091866706
deviation-center-line_min0.16085346197378936
deviation-heading_max14.341442554673254
deviation-heading_mean8.9926857991988
deviation-heading_median9.985540710640702
deviation-heading_min1.658219220840542
driven_any_max12.504811691058126
driven_any_mean9.051802596715646
driven_any_median11.415209357843866
driven_any_min0.8719799801167221
driven_lanedir_consec_max3.272602292300917
driven_lanedir_consec_mean2.11893701908899
driven_lanedir_consec_min0.4983131458885377
driven_lanedir_max3.986561316841666
driven_lanedir_mean3.058941507729533
driven_lanedir_median3.875445784093964
driven_lanedir_min0.4983131458885377
get_duckie_state_max1.4327821277436756e-06
get_duckie_state_mean1.3371813207939205e-06
get_duckie_state_median1.3094162762313955e-06
get_duckie_state_min1.297110602969215e-06
get_robot_state_max0.003789611501955768
get_robot_state_mean0.003692204400629383
get_robot_state_median0.003683966550671616
get_robot_state_min0.003611272999218532
get_state_dump_max0.004850977943057106
get_state_dump_mean0.004702370643009932
get_state_dump_median0.004680472647121407
get_state_dump_min0.004597559334739806
get_ui_image_max0.03544680504571824
get_ui_image_mean0.030581914270549886
get_ui_image_median0.03041258104131839
get_ui_image_min0.026055689953844514
in-drivable-lane_max36.74999999999862
in-drivable-lane_mean25.674999999999333
in-drivable-lane_min1.8999999999999932
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 10.335859846711813, "get_ui_image": 0.0277858842459936, "step_physics": 0.14976112331662858, "survival_time": 50.34999999999928, "driven_lanedir": 3.794325658971368, "get_state_dump": 0.004597559334739806, "get_robot_state": 0.003611272999218532, "sim_render-ego0": 0.0036777998246843857, "get_duckie_state": 1.297110602969215e-06, "in-drivable-lane": 29.09999999999932, "deviation-heading": 8.834101067648751, "agent_compute-ego0": 0.012340997656186422, "complete-iteration": 0.2153963077635992, "set_robot_commands": 0.0021657018907486445, "deviation-center-line": 1.66534631852761, "driven_lanedir_consec": 1.982370474032104, "sim_compute_sim_state": 0.009431421283691647, "sim_compute_performance-ego0": 0.0019392297854499212}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.8719799801167221, "get_ui_image": 0.03544680504571824, "step_physics": 0.22490136509849912, "survival_time": 5.1999999999999895, "driven_lanedir": 0.4983131458885377, "get_state_dump": 0.004850977943057106, "get_robot_state": 0.003702919823782784, "sim_render-ego0": 0.003787381308419364, "get_duckie_state": 1.4327821277436756e-06, "in-drivable-lane": 1.8999999999999932, "deviation-heading": 1.658219220840542, "agent_compute-ego0": 0.01272847538902646, "complete-iteration": 0.30018571217854817, "set_robot_commands": 0.0022381873357863653, "deviation-center-line": 0.16085346197378936, "driven_lanedir_consec": 0.4983131458885377, "sim_compute_sim_state": 0.010418878282819476, "sim_compute_performance-ego0": 0.002016487575712658}, "LF-norm-techtrack-000-ego0": {"driven_any": 12.49455886897592, "get_ui_image": 0.03303927783664319, "step_physics": 0.18817135019167375, "survival_time": 59.99999999999873, "driven_lanedir": 3.95656590921656, "get_state_dump": 0.004660759242150706, "get_robot_state": 0.003789611501955768, "sim_render-ego0": 0.003907566562878103, "get_duckie_state": 1.317555461696145e-06, "in-drivable-lane": 36.74999999999862, "deviation-heading": 11.136980353632657, "agent_compute-ego0": 0.012830386650155326, "complete-iteration": 0.26276402290814327, "set_robot_commands": 0.0022959951357877225, "deviation-center-line": 1.802346526798373, "driven_lanedir_consec": 3.272602292300917, "sim_compute_sim_state": 0.011883650890099417, "sim_compute_performance-ego0": 0.002095417218045529}, "LF-norm-small_loop-000-ego0": {"driven_any": 12.504811691058126, "get_ui_image": 0.026055689953844514, "step_physics": 0.14405771080004384, "survival_time": 59.99999999999873, "driven_lanedir": 3.986561316841666, "get_state_dump": 0.004700186052092109, "get_robot_state": 0.003665013277560448, "sim_render-ego0": 0.003734689072506513, "get_duckie_state": 1.3012770907666462e-06, "in-drivable-lane": 34.94999999999939, "deviation-heading": 14.341442554673254, "agent_compute-ego0": 0.01245860235577916, "complete-iteration": 0.205250157603217, "set_robot_commands": 0.0021854050451273923, "deviation-center-line": 2.09239612944691, "driven_lanedir_consec": 2.7224621641344005, "sim_compute_sim_state": 0.00635764124391478, "sim_compute_performance-ego0": 0.0019498464963914552}}
set_robot_commands_max0.0022959951357877225
set_robot_commands_mean0.0022213223518625313
set_robot_commands_median0.002211796190456879
set_robot_commands_min0.0021657018907486445
sim_compute_performance-ego0_max0.002095417218045529
sim_compute_performance-ego0_mean0.002000245268899891
sim_compute_performance-ego0_median0.0019831670360520564
sim_compute_performance-ego0_min0.0019392297854499212
sim_compute_sim_state_max0.011883650890099417
sim_compute_sim_state_mean0.00952289792513133
sim_compute_sim_state_median0.009925149783255562
sim_compute_sim_state_min0.00635764124391478
sim_render-ego0_max0.003907566562878103
sim_render-ego0_mean0.0037768591921220913
sim_render-ego0_median0.003761035190462938
sim_render-ego0_min0.0036777998246843857
simulation-passed1
step_physics_max0.22490136509849912
step_physics_mean0.17672288735171132
step_physics_median0.16896623675415118
step_physics_min0.14405771080004384
survival_time_max59.99999999999873
survival_time_mean43.88749999999918
survival_time_min5.1999999999999895
No reset possible
5428311624Anthony CourchesneΒ πŸ‡¨πŸ‡¦TNetR1aido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:43
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 35, in init
              ||     self.check_tensorflow_gpu()
              ||   File "solution.py", line 51, in check_tensorflow_gpu
              ||     raise Exception(msg)
              || Exception: Could not find gpu device.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 35, in init
              || |     self.check_tensorflow_gpu()
              || |   File "solution.py", line 51, in check_tensorflow_gpu
              || |     raise Exception(msg)
              || | Exception: Could not find gpu device.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 35, in init
              ||     self.check_tensorflow_gpu()
              ||   File "solution.py", line 51, in check_tensorflow_gpu
              ||     raise Exception(msg)
              || Exception: Could not find gpu device.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 35, in init
              || |     self.check_tensorflow_gpu()
              || |   File "solution.py", line 51, in check_tensorflow_gpu
              || |     raise Exception(msg)
              || | Exception: Could not find gpu device.
              || |
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5428111624Anthony CourchesneΒ πŸ‡¨πŸ‡¦TNetR1aido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:42
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 35, in init
              ||     self.check_tensorflow_gpu()
              ||   File "solution.py", line 51, in check_tensorflow_gpu
              ||     raise Exception(msg)
              || Exception: Could not find gpu device.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 35, in init
              || |     self.check_tensorflow_gpu()
              || |   File "solution.py", line 51, in check_tensorflow_gpu
              || |     raise Exception(msg)
              || | Exception: Could not find gpu device.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 35, in init
              ||     self.check_tensorflow_gpu()
              ||   File "solution.py", line 51, in check_tensorflow_gpu
              ||     raise Exception(msg)
              || Exception: Could not find gpu device.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 35, in init
              || |     self.check_tensorflow_gpu()
              || |   File "solution.py", line 51, in check_tensorflow_gpu
              || |     raise Exception(msg)
              || | Exception: Could not find gpu device.
              || |
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5416211646Melisande Tengexercise_state_estimationaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-030:39:54
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median5.0808039622633245
survival_time_median59.99999999999873
deviation-center-line_median2.931865503089304
in-drivable-lane_median27.799999999999187


other stats
agent_compute-ego0_max0.013526064127633883
agent_compute-ego0_mean0.013044796531674088
agent_compute-ego0_median0.012944245715621708
agent_compute-ego0_min0.012764630567819051
complete-iteration_max0.2737692807933671
complete-iteration_mean0.2476363195165749
complete-iteration_median0.2441083483048309
complete-iteration_min0.2285593006632707
deviation-center-line_max4.912833681163662
deviation-center-line_mean2.8078163576079405
deviation-center-line_min0.4547007430894923
deviation-heading_max20.5629428793135
deviation-heading_mean11.849618124248712
deviation-heading_median11.71428695163248
deviation-heading_min3.40695571441638
driven_any_max11.443761508982574
driven_any_mean10.547337647306517
driven_any_median10.66810109831776
driven_any_min9.409386883607969
driven_lanedir_consec_max9.084892015462264
driven_lanedir_consec_mean5.269025749546379
driven_lanedir_consec_min1.8296030581965992
driven_lanedir_max9.084892015462264
driven_lanedir_mean5.269025749546379
driven_lanedir_median5.0808039622633245
driven_lanedir_min1.8296030581965992
get_duckie_state_max1.569423134219467e-06
get_duckie_state_mean1.4944709763334676e-06
get_duckie_state_median1.4754755113841492e-06
get_duckie_state_min1.4575097483461049e-06
get_robot_state_max0.003939828410968558
get_robot_state_mean0.003822721684316033
get_robot_state_median0.003826076243938157
get_robot_state_min0.003698905838419258
get_state_dump_max0.005027824595607786
get_state_dump_mean0.004862780715059301
get_state_dump_median0.004832879175159
get_state_dump_min0.004757539914311418
get_ui_image_max0.03639613182518901
get_ui_image_mean0.03187469447852459
get_ui_image_median0.03207137460359799
get_ui_image_min0.02695989688171336
in-drivable-lane_max48.34999999999871
in-drivable-lane_mean28.03749999999923
in-drivable-lane_min8.199999999999847
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 10.562492962936515, "get_ui_image": 0.029975865241482182, "step_physics": 0.1700460359528785, "survival_time": 59.84999999999874, "driven_lanedir": 1.9384881464310084, "get_state_dump": 0.004848863525263256, "get_robot_state": 0.003939828410968558, "sim_render-ego0": 0.003973278458010971, "get_duckie_state": 1.569423134219467e-06, "in-drivable-lane": 48.34999999999871, "deviation-heading": 3.40695571441638, "agent_compute-ego0": 0.013526064127633883, "complete-iteration": 0.24147648743675623, "set_robot_commands": 0.0023361629556136855, "deviation-center-line": 0.4547007430894923, "driven_lanedir_consec": 1.9384881464310084, "sim_compute_sim_state": 0.010637232576666373, "sim_compute_performance-ego0": 0.002096476061316285}, "LF-norm-zigzag-000-ego0": {"driven_any": 10.773709233699003, "get_ui_image": 0.03639613182518901, "step_physics": 0.19427835514503752, "survival_time": 59.99999999999873, "driven_lanedir": 8.22311977809564, "get_state_dump": 0.004757539914311418, "get_robot_state": 0.003698905838419258, "sim_render-ego0": 0.0037533847815190424, "get_duckie_state": 1.4575097483461049e-06, "in-drivable-lane": 8.199999999999847, "deviation-heading": 20.5629428793135, "agent_compute-ego0": 0.01292568082912677, "complete-iteration": 0.2737692807933671, "set_robot_commands": 0.0021753595036134235, "deviation-center-line": 4.912833681163662, "driven_lanedir_consec": 8.22311977809564, "sim_compute_sim_state": 0.01367210031647567, "sim_compute_performance-ego0": 0.0020161588225733927}, "LF-norm-techtrack-000-ego0": {"driven_any": 11.443761508982574, "get_ui_image": 0.034166883965713796, "step_physics": 0.16929856069280544, "survival_time": 59.99999999999873, "driven_lanedir": 9.084892015462264, "get_state_dump": 0.004816894825054744, "get_robot_state": 0.0038586466834507416, "sim_render-ego0": 0.003917268273435365, "get_duckie_state": 1.477758453648652e-06, "in-drivable-lane": 8.399999999999753, "deviation-heading": 17.652827069873805, "agent_compute-ego0": 0.012962810602116645, "complete-iteration": 0.2467402091729055, "set_robot_commands": 0.0023211936569531495, "deviation-center-line": 4.641775429879741, "driven_lanedir_consec": 9.084892015462264, "sim_compute_sim_state": 0.013148791188502889, "sim_compute_performance-ego0": 0.0021516947226163053}, "LF-norm-small_loop-000-ego0": {"driven_any": 9.409386883607969, "get_ui_image": 0.02695989688171336, "step_physics": 0.16537602418269048, "survival_time": 59.99999999999873, "driven_lanedir": 1.8296030581965992, "get_state_dump": 0.005027824595607786, "get_robot_state": 0.003793505804425572, "sim_render-ego0": 0.0038307376150088345, "get_duckie_state": 1.4731925691196463e-06, "in-drivable-lane": 47.199999999998624, "deviation-heading": 5.775746833391158, "agent_compute-ego0": 0.012764630567819051, "complete-iteration": 0.2285593006632707, "set_robot_commands": 0.002311670015892518, "deviation-center-line": 1.2219555762988674, "driven_lanedir_consec": 1.8296030581965992, "sim_compute_sim_state": 0.006354296435722205, "sim_compute_performance-ego0": 0.002050363054680486}}
set_robot_commands_max0.0023361629556136855
set_robot_commands_mean0.0022860965330181943
set_robot_commands_median0.0023164318364228336
set_robot_commands_min0.0021753595036134235
sim_compute_performance-ego0_max0.0021516947226163053
sim_compute_performance-ego0_mean0.002078673165296617
sim_compute_performance-ego0_median0.0020734195579983857
sim_compute_performance-ego0_min0.0020161588225733927
sim_compute_sim_state_max0.01367210031647567
sim_compute_sim_state_mean0.010953105129341783
sim_compute_sim_state_median0.011893011882584629
sim_compute_sim_state_min0.006354296435722205
sim_render-ego0_max0.003973278458010971
sim_render-ego0_mean0.003868667281993553
sim_render-ego0_median0.0038740029442221
sim_render-ego0_min0.0037533847815190424
simulation-passed1
step_physics_max0.19427835514503752
step_physics_mean0.174749743993353
step_physics_median0.16967229832284195
step_physics_min0.16537602418269048
survival_time_max59.99999999999873
survival_time_mean59.96249999999873
survival_time_min59.84999999999874
No reset possible
5405811677Daniil Lisussim-exercise-2aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-030:36:07
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median9.856831938286811
survival_time_median58.999999999998785
deviation-center-line_median3.5737490212848293
in-drivable-lane_median17.124999999999517


other stats
agent_compute-ego0_max0.012519861396761756
agent_compute-ego0_mean0.012265038185314532
agent_compute-ego0_median0.012276261103640707
agent_compute-ego0_min0.01198776913721496
complete-iteration_max0.24091597024249
complete-iteration_mean0.20677026534018975
complete-iteration_median0.2042000099375487
complete-iteration_min0.17776507124317178
deviation-center-line_max4.094193358696928
deviation-center-line_mean3.6023178793565833
deviation-center-line_min3.1675801161597477
deviation-heading_max17.279119569747113
deviation-heading_mean13.41092561873364
deviation-heading_median13.990810692485931
deviation-heading_min8.38296152021558
driven_any_max17.955827366518772
driven_any_mean16.20292780009329
driven_any_median16.307726845962268
driven_any_min14.24043014192984
driven_lanedir_consec_max10.958719084296744
driven_lanedir_consec_mean9.426459239167556
driven_lanedir_consec_min7.0334539957998565
driven_lanedir_max11.823382042400226
driven_lanedir_mean10.9706089011205
driven_lanedir_median11.19841083185181
driven_lanedir_min9.662231898378147
get_duckie_state_max1.3594424893318069e-06
get_duckie_state_mean1.2988236460857572e-06
get_duckie_state_median1.3030895050163105e-06
get_duckie_state_min1.2296730849786011e-06
get_robot_state_max0.003822162720240748
get_robot_state_mean0.003768259958356616
get_robot_state_median0.00377207426247562
get_robot_state_min0.003706728588234475
get_state_dump_max0.00480831572654146
get_state_dump_mean0.004749419851233922
get_state_dump_median0.004756483637157636
get_state_dump_min0.004676396404078958
get_ui_image_max0.036010728291329695
get_ui_image_mean0.03135098437778899
get_ui_image_median0.03125425751851149
get_ui_image_min0.02688469418280329
in-drivable-lane_max20.549999999999045
in-drivable-lane_mean17.4124999999994
in-drivable-lane_min14.849999999999522
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 16.088563211328694, "get_ui_image": 0.02938694315529912, "step_physics": 0.12552007517093736, "survival_time": 54.84999999999902, "driven_lanedir": 10.958719084296744, "get_state_dump": 0.004728511165834298, "get_robot_state": 0.003822162720240748, "sim_render-ego0": 0.003918814963113197, "get_duckie_state": 1.3477815304948984e-06, "in-drivable-lane": 20.549999999999045, "deviation-heading": 8.38296152021558, "agent_compute-ego0": 0.012519861396761756, "complete-iteration": 0.19422258162541903, "set_robot_commands": 0.00230588113892491, "deviation-center-line": 3.1675801161597477, "driven_lanedir_consec": 10.958719084296744, "sim_compute_sim_state": 0.009815353513415394, "sim_compute_performance-ego0": 0.002117325047977635}, "LF-norm-zigzag-000-ego0": {"driven_any": 14.24043014192984, "get_ui_image": 0.036010728291329695, "step_physics": 0.16368201610746233, "survival_time": 59.99999999999873, "driven_lanedir": 9.662231898378147, "get_state_dump": 0.004676396404078958, "get_robot_state": 0.003706728588234475, "sim_render-ego0": 0.003806644434932865, "get_duckie_state": 1.2583974795377223e-06, "in-drivable-lane": 17.899999999999658, "deviation-heading": 14.221835292121682, "agent_compute-ego0": 0.012039538724138576, "complete-iteration": 0.24091597024249, "set_robot_commands": 0.0022063237443553916, "deviation-center-line": 3.4956481580295056, "driven_lanedir_consec": 7.0334539957998565, "sim_compute_sim_state": 0.012672878324141807, "sim_compute_performance-ego0": 0.0020292644993053885}, "LF-norm-techtrack-000-ego0": {"driven_any": 16.526890480595842, "get_ui_image": 0.033121571881723856, "step_physics": 0.1394150012736686, "survival_time": 57.99999999999884, "driven_lanedir": 11.823382042400226, "get_state_dump": 0.0047844561084809745, "get_robot_state": 0.003783376539297703, "sim_render-ego0": 0.003866312731760109, "get_duckie_state": 1.2296730849786011e-06, "in-drivable-lane": 14.849999999999522, "deviation-heading": 13.759786092850185, "agent_compute-ego0": 0.012512983483142837, "complete-iteration": 0.21417743824967836, "set_robot_commands": 0.0022644269682024184, "deviation-center-line": 3.651849884540153, "driven_lanedir_consec": 10.570152765798372, "sim_compute_sim_state": 0.012256112620298664, "sim_compute_performance-ego0": 0.002086311245046742}, "LF-norm-small_loop-000-ego0": {"driven_any": 17.955827366518772, "get_ui_image": 0.02688469418280329, "step_physics": 0.11589581027416068, "survival_time": 59.99999999999873, "driven_lanedir": 11.438102579406875, "get_state_dump": 0.00480831572654146, "get_robot_state": 0.0037607719856535366, "sim_render-ego0": 0.003787457794075107, "get_duckie_state": 1.3594424893318069e-06, "in-drivable-lane": 16.349999999999373, "deviation-heading": 17.279119569747113, "agent_compute-ego0": 0.01198776913721496, "complete-iteration": 0.17776507124317178, "set_robot_commands": 0.00226410640268699, "deviation-center-line": 4.094193358696928, "driven_lanedir_consec": 9.143511110775254, "sim_compute_sim_state": 0.006283006302820058, "sim_compute_performance-ego0": 0.002009218678089304}}
set_robot_commands_max0.00230588113892491
set_robot_commands_mean0.0022601845635424273
set_robot_commands_median0.0022642666854447045
set_robot_commands_min0.0022063237443553916
sim_compute_performance-ego0_max0.002117325047977635
sim_compute_performance-ego0_mean0.0020605298676047673
sim_compute_performance-ego0_median0.002057787872176065
sim_compute_performance-ego0_min0.002009218678089304
sim_compute_sim_state_max0.012672878324141807
sim_compute_sim_state_mean0.010256837690168982
sim_compute_sim_state_median0.01103573306685703
sim_compute_sim_state_min0.006283006302820058
sim_render-ego0_max0.003918814963113197
sim_render-ego0_mean0.003844807480970319
sim_render-ego0_median0.0038364785833464865
sim_render-ego0_min0.003787457794075107
simulation-passed1
step_physics_max0.16368201610746233
step_physics_mean0.13612822570655722
step_physics_median0.13246753822230298
step_physics_min0.11589581027416068
survival_time_max59.99999999999873
survival_time_mean58.212499999998826
survival_time_min54.84999999999902
No reset possible
5395711315Anastasiya NikolskayaΒ πŸ‡·πŸ‡ΊJetBrains Researchaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-030:46:32
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.9575220411172665
survival_time_median59.99999999999873
deviation-center-line_median5.92470691520634
in-drivable-lane_median22.29999999999931


other stats
agent_compute-ego0_max0.058073004715448615
agent_compute-ego0_mean0.053901372751923624
agent_compute-ego0_median0.054728734205406374
agent_compute-ego0_min0.04807501788143314
complete-iteration_max0.40718666838170287
complete-iteration_mean0.3392494259825555
complete-iteration_median0.3314262011168303
complete-iteration_min0.2869586333148585
deviation-center-line_max10.693228045124844
deviation-center-line_mean5.922064322129089
deviation-center-line_min1.1456154129788352
deviation-heading_max18.41106986958364
deviation-heading_mean10.426591609862394
deviation-heading_median9.863911646988798
deviation-heading_min3.567473275888339
driven_any_max7.622248692683846
driven_any_mean3.876328241538314
driven_any_median3.5058278144877546
driven_any_min0.8714086444939001
driven_lanedir_consec_max7.005153690969786
driven_lanedir_consec_mean2.9433237169902684
driven_lanedir_consec_min0.8530970947567543
driven_lanedir_max7.005153690969786
driven_lanedir_mean2.9433237169902684
driven_lanedir_median1.9575220411172665
driven_lanedir_min0.8530970947567543
get_duckie_state_max1.4388491767927769e-06
get_duckie_state_mean1.3826689454141405e-06
get_duckie_state_median1.3916021977534995e-06
get_duckie_state_min1.308622209356786e-06
get_robot_state_max0.003985581251107882
get_robot_state_mean0.0038773195630406264
get_robot_state_median0.003865970858527064
get_robot_state_min0.0037917552840004954
get_state_dump_max0.005024684259635423
get_state_dump_mean0.0049234545796638125
get_state_dump_median0.004905293625062153
get_state_dump_min0.004858546808895521
get_ui_image_max0.036876817031466495
get_ui_image_mean0.03137092005700295
get_ui_image_median0.03079933955409346
get_ui_image_min0.027008184088358377
in-drivable-lane_max48.69999999999865
in-drivable-lane_mean23.32499999999932
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 7.622248692683846, "get_ui_image": 0.02868656631711123, "step_physics": 0.1929644779996213, "survival_time": 59.99999999999873, "driven_lanedir": 7.005153690969786, "get_state_dump": 0.004858546808895521, "get_robot_state": 0.003802539109191133, "sim_render-ego0": 0.003907530432835308, "get_duckie_state": 1.4388491767927769e-06, "in-drivable-lane": 43.449999999998624, "deviation-heading": 3.567473275888339, "agent_compute-ego0": 0.052054819318277455, "complete-iteration": 0.299485627062414, "set_robot_commands": 0.0022482195067266737, "deviation-center-line": 1.3287891977930597, "driven_lanedir_consec": 7.005153690969786, "sim_compute_sim_state": 0.008816451454639038, "sim_compute_performance-ego0": 0.0020547512667462987}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.6357525086111688, "get_ui_image": 0.036876817031466495, "step_physics": 0.2832996182199521, "survival_time": 59.99999999999873, "driven_lanedir": 1.161891071053795, "get_state_dump": 0.004907382715750892, "get_robot_state": 0.003985581251107882, "sim_render-ego0": 0.0041159197849397556, "get_duckie_state": 1.394579948533286e-06, "in-drivable-lane": 1.149999999999996, "deviation-heading": 18.41106986958364, "agent_compute-ego0": 0.058073004715448615, "complete-iteration": 0.40718666838170287, "set_robot_commands": 0.0024047600637367625, "deviation-center-line": 10.52062463261962, "driven_lanedir_consec": 1.161891071053795, "sim_compute_sim_state": 0.011228628102984656, "sim_compute_performance-ego0": 0.002203234625696441}, "LF-norm-techtrack-000-ego0": {"driven_any": 0.8714086444939001, "get_ui_image": 0.03291211279107569, "step_physics": 0.2542087938465941, "survival_time": 59.99999999999873, "driven_lanedir": 0.8530970947567543, "get_state_dump": 0.005024684259635423, "get_robot_state": 0.003929402607862995, "sim_render-ego0": 0.004045929738028063, "get_duckie_state": 1.3886244469737134e-06, "in-drivable-lane": 0.0, "deviation-heading": 13.692242218739208, "agent_compute-ego0": 0.04807501788143314, "complete-iteration": 0.36336677517124655, "set_robot_commands": 0.0023541222206261833, "deviation-center-line": 10.693228045124844, "driven_lanedir_consec": 0.8530970947567543, "sim_compute_sim_state": 0.010591227843501388, "sim_compute_performance-ego0": 0.0021304375523830037}, "LF-norm-small_loop-000-ego0": {"driven_any": 5.37590312036434, "get_ui_image": 0.027008184088358377, "step_physics": 0.17890472475634725, "survival_time": 59.99999999999873, "driven_lanedir": 2.7531530111807374, "get_state_dump": 0.004903204534373414, "get_robot_state": 0.0037917552840004954, "sim_render-ego0": 0.00396475565622093, "get_duckie_state": 1.308622209356786e-06, "in-drivable-lane": 48.69999999999865, "deviation-heading": 6.03558107523839, "agent_compute-ego0": 0.05740264909253529, "complete-iteration": 0.2869586333148585, "set_robot_commands": 0.002307380466635876, "deviation-center-line": 1.1456154129788352, "driven_lanedir_consec": 2.7531530111807374, "sim_compute_sim_state": 0.006538410766436397, "sim_compute_performance-ego0": 0.002047627097264019}}
set_robot_commands_max0.0024047600637367625
set_robot_commands_mean0.002328620564431374
set_robot_commands_median0.0023307513436310297
set_robot_commands_min0.0022482195067266737
sim_compute_performance-ego0_max0.002203234625696441
sim_compute_performance-ego0_mean0.0021090126355224406
sim_compute_performance-ego0_median0.002092594409564651
sim_compute_performance-ego0_min0.002047627097264019
sim_compute_sim_state_max0.011228628102984656
sim_compute_sim_state_mean0.009293679541890369
sim_compute_sim_state_median0.009703839649070211
sim_compute_sim_state_min0.006538410766436397
sim_render-ego0_max0.0041159197849397556
sim_render-ego0_mean0.004008533903006014
sim_render-ego0_median0.004005342697124497
sim_render-ego0_min0.003907530432835308
simulation-passed1
step_physics_max0.2832996182199521
step_physics_mean0.22734440370562867
step_physics_median0.2235866359231077
step_physics_min0.17890472475634725
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
5395111704brian wongtemplate-randomaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:01:29
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 53, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 777, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 991, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 120, in write_logs
    services2id: Dict[ServiceName, ContainerID] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 63, in get_services_id
    raise DockerComposeFail(msg, output=output.decode(), names=names) from e
duckietown_challenges_runner.docker_compose.DockerComposeFail: Cannot get process ids
β”‚ output: ''
β”‚  names: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5394011733Jean-SΓ©bastien GrondinΒ πŸ‡¨πŸ‡¦sim-exercise-2aido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:02:01
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 53, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 777, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 991, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 120, in write_logs
    services2id: Dict[ServiceName, ContainerID] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 63, in get_services_id
    raise DockerComposeFail(msg, output=output.decode(), names=names) from e
duckietown_challenges_runner.docker_compose.DockerComposeFail: Cannot get process ids
β”‚ output: ''
β”‚  names: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible