Duckietown Challenges Home Challenges Submissions

Evaluator 4883

ID4883
evaluatornogpu-prod-04
ownerI don't have one 😀
machinenogpu-prod_af4c8b961571
processnogpu-prod-04_af4c8b961571
version6.2.5
first heard
last heard
statusinactive
# evaluating
# success18 53908
# timeout2 56493
# failed
# error1 54087
# aborted3 57179
# host-error39 54304
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
5757410891Ayman Shams 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simtimeoutyesnogpu-prod-04----No reset possible
5755510933Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simabortedyesnogpu-prod-040:03:15
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 45, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 45, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 45, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 45, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5739910942Ayman Shams 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-040:42:26
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.0
survival_time_median59.99999999999873
deviation-center-line_median1.2422730096440104
in-drivable-lane_median0.0


other stats
agent_compute-ego0_max0.013062841389995134
agent_compute-ego0_mean0.012267349562378946
agent_compute-ego0_median0.012303291510582764
agent_compute-ego0_min0.011399973838355122
complete-iteration_max0.3459626064014673
complete-iteration_mean0.2960075480455562
complete-iteration_median0.2918289627858145
complete-iteration_min0.2544096602091285
deviation-center-line_max4.053503393024394
deviation-center-line_mean1.731069028434027
deviation-center-line_min0.386226701423694
deviation-heading_max27.859809596736422
deviation-heading_mean14.925250437835436
deviation-heading_median14.309269950178932
deviation-heading_min3.22265225424745
driven_any_max2.6645352591003757e-13
driven_any_mean1.9984014443252818e-13
driven_any_median2.6645352591003757e-13
driven_any_min0.0
driven_lanedir_consec_max0.000286102294921875
driven_lanedir_consec_mean7.152557373046875e-05
driven_lanedir_consec_min0.0
driven_lanedir_max0.000286102294921875
driven_lanedir_mean7.152557373046875e-05
driven_lanedir_median0.0
driven_lanedir_min0.0
get_duckie_state_max1.49721309207659e-06
get_duckie_state_mean1.4112057237204268e-06
get_duckie_state_median1.418302696412251e-06
get_duckie_state_min1.3110044099806152e-06
get_robot_state_max0.003871170507680367
get_robot_state_mean0.003717429532695074
get_robot_state_median0.003708521491978984
get_robot_state_min0.0035815046391419626
get_state_dump_max0.00477945258674971
get_state_dump_mean0.0046977081167807085
get_state_dump_median0.004699231782225546
get_state_dump_min0.004612916315922034
get_ui_image_max0.03544453042035893
get_ui_image_mean0.030744707802749495
get_ui_image_median0.029921524034352425
get_ui_image_min0.027691252721934196
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 2.6645352591003757e-13, "get_ui_image": 0.027691252721934196, "step_physics": 0.2068831745929067, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "get_state_dump": 0.004657848391505105, "get_robot_state": 0.0035815046391419626, "sim_render-ego0": 0.0037496389695547898, "get_duckie_state": 1.449569079600008e-06, "in-drivable-lane": 0.0, "deviation-heading": 22.66279310353771, "agent_compute-ego0": 0.011399973838355122, "complete-iteration": 0.2704767609119018, "set_robot_commands": 0.002179997052677863, "deviation-center-line": 4.053503393024394, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.008337911022989876, "sim_compute_performance-ego0": 0.0019153296004524832}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.0, "get_ui_image": 0.03544453042035893, "step_physics": 0.2693547219856891, "survival_time": 59.99999999999873, "driven_lanedir": 0.000286102294921875, "get_state_dump": 0.00477945258674971, "get_robot_state": 0.0036779252019750385, "sim_render-ego0": 0.0038920499005980734, "get_duckie_state": 1.3110044099806152e-06, "in-drivable-lane": 0.0, "deviation-heading": 27.859809596736422, "agent_compute-ego0": 0.01295041482117055, "complete-iteration": 0.3459626064014673, "set_robot_commands": 0.0021968979720370556, "deviation-center-line": 1.0457540566888746, "driven_lanedir_consec": 0.000286102294921875, "sim_compute_sim_state": 0.011596825994321647, "sim_compute_performance-ego0": 0.0019881217108479545}, "LF-norm-techtrack-000-ego0": {"driven_any": 2.6645352591003757e-13, "get_ui_image": 0.03213658479727079, "step_physics": 0.24376661870799196, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "get_state_dump": 0.004612916315922034, "get_robot_state": 0.00373911778198293, "sim_render-ego0": 0.003895538236378234, "get_duckie_state": 1.49721309207659e-06, "in-drivable-lane": 0.0, "deviation-heading": 3.22265225424745, "agent_compute-ego0": 0.01165616819999498, "complete-iteration": 0.31318116465972723, "set_robot_commands": 0.0021433268458916684, "deviation-center-line": 1.4387919625991463, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.009080597998994673, "sim_compute_performance-ego0": 0.0020583890856156043}, "LF-norm-small_loop-000-ego0": {"driven_any": 2.6645352591003757e-13, "get_ui_image": 0.027706463271434063, "step_physics": 0.19046693618450436, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "get_state_dump": 0.004740615172945987, "get_robot_state": 0.003871170507680367, "sim_render-ego0": 0.003963057742726297, "get_duckie_state": 1.387036313224494e-06, "in-drivable-lane": 0.0, "deviation-heading": 5.955746796820156, "agent_compute-ego0": 0.013062841389995134, "complete-iteration": 0.2544096602091285, "set_robot_commands": 0.0022503638843215574, "deviation-center-line": 0.386226701423694, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.006249781750719514, "sim_compute_performance-ego0": 0.0020128650331775115}}
set_robot_commands_max0.0022503638843215574
set_robot_commands_mean0.0021926464387320363
set_robot_commands_median0.002188447512357459
set_robot_commands_min0.0021433268458916684
sim_compute_performance-ego0_max0.0020583890856156043
sim_compute_performance-ego0_mean0.0019936763575233887
sim_compute_performance-ego0_median0.002000493372012733
sim_compute_performance-ego0_min0.0019153296004524832
sim_compute_sim_state_max0.011596825994321647
sim_compute_sim_state_mean0.008816279191756426
sim_compute_sim_state_median0.008709254510992274
sim_compute_sim_state_min0.006249781750719514
sim_render-ego0_max0.003963057742726297
sim_render-ego0_mean0.003875071212314348
sim_render-ego0_median0.0038937940684881534
sim_render-ego0_min0.0037496389695547898
simulation-passed1
step_physics_max0.2693547219856891
step_physics_mean0.22761786286777305
step_physics_median0.22532489665044933
step_physics_min0.19046693618450436
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
5737311010Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simabortedyesnogpu-prod-040:02:47
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 45, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 45, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 45, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 45, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5728010980Fernanda Custodio Pereira do Carmo 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-040:21:21
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median4.654618826563127
survival_time_median38.04999999999978
deviation-center-line_median1.3962319994832029
in-drivable-lane_median12.874999999999645


other stats
agent_compute-ego0_max0.040835550439502544
agent_compute-ego0_mean0.03724212952602486
agent_compute-ego0_median0.038999604554279046
agent_compute-ego0_min0.030133758556038812
complete-iteration_max0.2139413762189116
complete-iteration_mean0.188871919143701
complete-iteration_median0.1839080326663501
complete-iteration_min0.17373023502319213
deviation-center-line_max3.700738831343764
deviation-center-line_mean1.8162354212065897
deviation-center-line_min0.7717388545161897
deviation-heading_max7.771859218641016
deviation-heading_mean3.893281181120575
deviation-heading_median3.1635858940602812
deviation-heading_min1.4740937177207212
driven_any_max11.330327077043858
driven_any_mean6.886646690918119
driven_any_median7.038027771084406
driven_any_min2.140204144459806
driven_lanedir_consec_max6.9609594130439305
driven_lanedir_consec_mean4.494687689535361
driven_lanedir_consec_min1.7085536919712574
driven_lanedir_max6.9609594130439305
driven_lanedir_mean4.494687689535361
driven_lanedir_median4.654618826563127
driven_lanedir_min1.7085536919712574
get_duckie_state_max1.3999398999369016e-06
get_duckie_state_mean1.2555295889408078e-06
get_duckie_state_median1.2594446208016549e-06
get_duckie_state_min1.10328921422302e-06
get_robot_state_max0.003816755089930551
get_robot_state_mean0.003560721255722305
get_robot_state_median0.003528258358966525
get_robot_state_min0.0033696132150256197
get_state_dump_max0.004844008635521729
get_state_dump_mean0.004582507808371453
get_state_dump_median0.004585177886873407
get_state_dump_min0.0043156668242172675
get_ui_image_max0.03213575108331225
get_ui_image_mean0.028897052380048836
get_ui_image_median0.029003871051030698
get_ui_image_min0.025444716334821708
in-drivable-lane_max21.24999999999946
in-drivable-lane_mean12.449999999999696
in-drivable-lane_min2.8000000000000353
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 4.812496026221973, "get_ui_image": 0.02634181940194332, "step_physics": 0.08481536534699527, "survival_time": 26.35000000000024, "driven_lanedir": 3.987910884151872, "get_state_dump": 0.00436451489275152, "get_robot_state": 0.003427836931113041, "sim_render-ego0": 0.003649142655459317, "get_duckie_state": 1.240860332142223e-06, "in-drivable-lane": 4.90000000000007, "deviation-heading": 1.6468742533219165, "agent_compute-ego0": 0.038178570342786385, "complete-iteration": 0.17451408369974655, "set_robot_commands": 0.0020791504419211187, "deviation-center-line": 0.7717388545161897, "driven_lanedir_consec": 3.987910884151872, "sim_compute_sim_state": 0.009726865725083784, "sim_compute_performance-ego0": 0.0018523452859936335}, "LF-norm-zigzag-000-ego0": {"driven_any": 2.140204144459806, "get_ui_image": 0.03213575108331225, "step_physics": 0.11592874642808428, "survival_time": 12.30000000000004, "driven_lanedir": 1.7085536919712574, "get_state_dump": 0.0043156668242172675, "get_robot_state": 0.0033696132150256197, "sim_render-ego0": 0.0036053966414107967, "get_duckie_state": 1.10328921422302e-06, "in-drivable-lane": 2.8000000000000353, "deviation-heading": 1.4740937177207212, "agent_compute-ego0": 0.040835550439502544, "complete-iteration": 0.2139413762189116, "set_robot_commands": 0.0020203078806641612, "deviation-center-line": 0.8808883417988647, "driven_lanedir_consec": 1.7085536919712574, "sim_compute_sim_state": 0.009884600697258706, "sim_compute_performance-ego0": 0.001767895964958407}, "LF-norm-techtrack-000-ego0": {"driven_any": 11.330327077043858, "get_ui_image": 0.03166592270011807, "step_physics": 0.10239240291414411, "survival_time": 59.99999999999873, "driven_lanedir": 6.9609594130439305, "get_state_dump": 0.004844008635521729, "get_robot_state": 0.003816755089930551, "sim_render-ego0": 0.004032407374703616, "get_duckie_state": 1.3999398999369016e-06, "in-drivable-lane": 21.24999999999946, "deviation-heading": 7.771859218641016, "agent_compute-ego0": 0.030133758556038812, "complete-iteration": 0.19330198163295367, "set_robot_commands": 0.0023272351162518987, "deviation-center-line": 3.700738831343764, "driven_lanedir_consec": 6.9609594130439305, "sim_compute_sim_state": 0.01193663917115884, "sim_compute_performance-ego0": 0.0020585923667355045}, "LF-norm-small_loop-000-ego0": {"driven_any": 9.263559515946838, "get_ui_image": 0.025444716334821708, "step_physics": 0.08607032141053533, "survival_time": 49.74999999999931, "driven_lanedir": 5.321326768974384, "get_state_dump": 0.0048058408809952945, "get_robot_state": 0.0036286797868200094, "sim_render-ego0": 0.0038131463479804226, "get_duckie_state": 1.2780289094610864e-06, "in-drivable-lane": 20.84999999999922, "deviation-heading": 4.680297534798646, "agent_compute-ego0": 0.03982063876577171, "complete-iteration": 0.17373023502319213, "set_robot_commands": 0.0022577454287364303, "deviation-center-line": 1.911575657167541, "driven_lanedir_consec": 5.321326768974384, "sim_compute_sim_state": 0.005875195844106406, "sim_compute_performance-ego0": 0.00193348395297805}}
set_robot_commands_max0.0023272351162518987
set_robot_commands_mean0.0021711097168934023
set_robot_commands_median0.0021684479353287745
set_robot_commands_min0.0020203078806641612
sim_compute_performance-ego0_max0.0020585923667355045
sim_compute_performance-ego0_mean0.0019030793926663988
sim_compute_performance-ego0_median0.0018929146194858416
sim_compute_performance-ego0_min0.001767895964958407
sim_compute_sim_state_max0.01193663917115884
sim_compute_sim_state_mean0.009355825359401932
sim_compute_sim_state_median0.009805733211171244
sim_compute_sim_state_min0.005875195844106406
sim_render-ego0_max0.004032407374703616
sim_render-ego0_mean0.003775023254888538
sim_render-ego0_median0.00373114450171987
sim_render-ego0_min0.0036053966414107967
simulation-passed1
step_physics_max0.11592874642808428
step_physics_mean0.09730170902493974
step_physics_median0.09423136216233972
step_physics_min0.08481536534699527
survival_time_max59.99999999999873
survival_time_mean37.09999999999958
survival_time_min12.30000000000004
No reset possible
5719810994Mo Kleit 🇨🇦sim-exercise-1aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-040:23:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.7516836624689902
survival_time_median39.224999999999426
deviation-center-line_median1.1374611116559985
in-drivable-lane_median9.60000000000012


other stats
agent_compute-ego0_max0.013111895943289676
agent_compute-ego0_mean0.012792874708905964
agent_compute-ego0_median0.012842292888873223
agent_compute-ego0_min0.012375017114587732
complete-iteration_max0.1884661284612676
complete-iteration_mean0.17024317088622176
complete-iteration_median0.16820816285307932
complete-iteration_min0.15609022937746073
deviation-center-line_max4.408665267074139
deviation-center-line_mean1.8253448791423856
deviation-center-line_min0.6177920261834066
deviation-heading_max10.73000726254849
deviation-heading_mean4.39934653655751
deviation-heading_median2.613259236683663
deviation-heading_min1.640860410314223
driven_any_max6.618033478166794
driven_any_mean3.732259003446574
driven_any_median3.508242416385705
driven_any_min1.2945177028480936
driven_lanedir_consec_max6.297743214185447
driven_lanedir_consec_mean2.657459158647139
driven_lanedir_consec_min0.8287260954651285
driven_lanedir_max6.297743214185447
driven_lanedir_mean2.657459158647139
driven_lanedir_median1.7516836624689902
driven_lanedir_min0.8287260954651285
get_duckie_state_max1.39215391141007e-06
get_duckie_state_mean1.3339054993782906e-06
get_duckie_state_median1.3368775768010906e-06
get_duckie_state_min1.2697129325009107e-06
get_robot_state_max0.003731898920025853
get_robot_state_mean0.0036394595275333814
get_robot_state_median0.0036443654324768
get_robot_state_min0.0035372083251540725
get_state_dump_max0.0047113760036624945
get_state_dump_mean0.004569157656652017
get_state_dump_median0.004559582516513796
get_state_dump_min0.004446089589917982
get_ui_image_max0.03464634293422016
get_ui_image_mean0.029687782652702197
get_ui_image_median0.02898846785220682
get_ui_image_min0.026127851972174983
in-drivable-lane_max37.64999999999854
in-drivable-lane_mean14.362499999999688
in-drivable-lane_min0.5999999999999659
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 1.7216128615304926, "get_ui_image": 0.026428149841927193, "step_physics": 0.09355018783260036, "survival_time": 18.450000000000127, "driven_lanedir": 0.884215966895221, "get_state_dump": 0.004446089589917982, "get_robot_state": 0.0035372083251540725, "sim_render-ego0": 0.003700459325635755, "get_duckie_state": 1.3248340503589527e-06, "in-drivable-lane": 12.20000000000014, "deviation-heading": 1.640860410314223, "agent_compute-ego0": 0.012375017114587732, "complete-iteration": 0.15695315953847525, "set_robot_commands": 0.002089231723063701, "deviation-center-line": 0.6630741433797039, "driven_lanedir_consec": 0.884215966895221, "sim_compute_sim_state": 0.00888031108959301, "sim_compute_performance-ego0": 0.0018608164142917944}, "LF-norm-zigzag-000-ego0": {"driven_any": 6.618033478166794, "get_ui_image": 0.03464634293422016, "step_physics": 0.11278683299525988, "survival_time": 59.99999999999873, "driven_lanedir": 6.297743214185447, "get_state_dump": 0.004530538825766431, "get_robot_state": 0.003613785244245315, "sim_render-ego0": 0.003893471875853781, "get_duckie_state": 1.2697129325009107e-06, "in-drivable-lane": 0.5999999999999659, "deviation-heading": 10.73000726254849, "agent_compute-ego0": 0.012824751753096376, "complete-iteration": 0.1884661284612676, "set_robot_commands": 0.002177693266157902, "deviation-center-line": 4.408665267074139, "driven_lanedir_consec": 6.297743214185447, "sim_compute_sim_state": 0.011947048792334818, "sim_compute_performance-ego0": 0.001956574823536742}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.2945177028480936, "get_ui_image": 0.031548785862486446, "step_physics": 0.10918433658708156, "survival_time": 15.80000000000009, "driven_lanedir": 0.8287260954651285, "get_state_dump": 0.0047113760036624945, "get_robot_state": 0.003674945620708285, "sim_render-ego0": 0.003932422643956325, "get_duckie_state": 1.39215391141007e-06, "in-drivable-lane": 7.0000000000000995, "deviation-heading": 1.7899645865531544, "agent_compute-ego0": 0.013111895943289676, "complete-iteration": 0.1794631661676834, "set_robot_commands": 0.0022171739524095217, "deviation-center-line": 0.6177920261834066, "driven_lanedir_consec": 0.8287260954651285, "sim_compute_sim_state": 0.009005117115538202, "sim_compute_performance-ego0": 0.0019824008460300577}, "LF-norm-small_loop-000-ego0": {"driven_any": 5.294871971240918, "get_ui_image": 0.026127851972174983, "step_physics": 0.09422843164448734, "survival_time": 59.99999999999873, "driven_lanedir": 2.619151358042759, "get_state_dump": 0.004588626207261161, "get_robot_state": 0.003731898920025853, "sim_render-ego0": 0.00389885485519676, "get_duckie_state": 1.3489211032432284e-06, "in-drivable-lane": 37.64999999999854, "deviation-heading": 3.4365538868141723, "agent_compute-ego0": 0.012859834024650072, "complete-iteration": 0.15609022937746073, "set_robot_commands": 0.002299629182839374, "deviation-center-line": 1.6118480799322925, "driven_lanedir_consec": 2.619151358042759, "sim_compute_sim_state": 0.0062374506862237, "sim_compute_performance-ego0": 0.002026560900114061}}
set_robot_commands_max0.002299629182839374
set_robot_commands_mean0.0021959320311176244
set_robot_commands_median0.0021974336092837115
set_robot_commands_min0.002089231723063701
sim_compute_performance-ego0_max0.002026560900114061
sim_compute_performance-ego0_mean0.001956588245993164
sim_compute_performance-ego0_median0.0019694878347834
sim_compute_performance-ego0_min0.0018608164142917944
sim_compute_sim_state_max0.011947048792334818
sim_compute_sim_state_mean0.009017481920922433
sim_compute_sim_state_median0.008942714102565606
sim_compute_sim_state_min0.0062374506862237
sim_render-ego0_max0.003932422643956325
sim_render-ego0_mean0.003856302175160656
sim_render-ego0_median0.0038961633655252704
sim_render-ego0_min0.003700459325635755
simulation-passed1
step_physics_max0.11278683299525988
step_physics_mean0.10243744726485728
step_physics_median0.10170638411578443
step_physics_min0.09355018783260036
survival_time_max59.99999999999873
survival_time_mean38.56249999999942
survival_time_min15.80000000000009
No reset possible
5717911012Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simabortedyesnogpu-prod-040:02:58
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5715611313Moustafa Elarabitemplate-pytorchaido5-LFV_multi-sim-validationLFVmultibodyv-simhost-erroryesnogpu-prod-040:03:51
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5698611047Bhavya Patwa 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-040:28:02
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median2.384268564004464
survival_time_median56.84999999999891
deviation-center-line_median1.6714619089412763
in-drivable-lane_median13.099999999999785


other stats
agent_compute-ego0_max0.01276808073354322
agent_compute-ego0_mean0.012352274743181155
agent_compute-ego0_median0.01242892172414
agent_compute-ego0_min0.011783174790901407
complete-iteration_max0.197470470251039
complete-iteration_mean0.18582743766362217
complete-iteration_median0.1898849344433323
complete-iteration_min0.16606941151678511
deviation-center-line_max2.9837212370469395
deviation-center-line_mean1.698670008439766
deviation-center-line_min0.46803497882957223
deviation-heading_max13.362777940549266
deviation-heading_mean9.2835351537956
deviation-heading_median9.947838569369464
deviation-heading_min3.875685535894214
driven_any_max6.296564739392844
driven_any_mean4.38183524644512
driven_any_median4.909881723516703
driven_any_min1.411012799354233
driven_lanedir_consec_max5.952690535180785
driven_lanedir_consec_mean2.809817067748705
driven_lanedir_consec_min0.5180406078051067
driven_lanedir_max5.952690535180785
driven_lanedir_mean2.809817067748705
driven_lanedir_median2.384268564004464
driven_lanedir_min0.5180406078051067
get_duckie_state_max2.1269155103106833e-06
get_duckie_state_mean2.005173451164561e-06
get_duckie_state_median2.010882818727596e-06
get_duckie_state_min1.8720126568923685e-06
get_robot_state_max0.0037508445514750546
get_robot_state_mean0.003662071833514475
get_robot_state_median0.003669155191041428
get_robot_state_min0.0035591324004999904
get_state_dump_max0.004772864718769872
get_state_dump_mean0.0046202441382287305
get_state_dump_median0.004643485602330079
get_state_dump_min0.00442114062948489
get_ui_image_max0.03401057448216421
get_ui_image_mean0.029700189531614907
get_ui_image_median0.029441600686646557
get_ui_image_min0.025906982271002296
in-drivable-lane_max37.19999999999885
in-drivable-lane_mean16.07499999999961
in-drivable-lane_min0.9000000000000128
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 1.411012799354233, "get_ui_image": 0.027135266056009657, "step_physics": 0.1188898847186214, "survival_time": 18.60000000000013, "driven_lanedir": 0.5180406078051067, "get_state_dump": 0.0046111489109315435, "get_robot_state": 0.0037508445514750546, "sim_render-ego0": 0.003863670231509784, "get_duckie_state": 1.916297319427572e-06, "in-drivable-lane": 10.250000000000146, "deviation-heading": 3.875685535894214, "agent_compute-ego0": 0.011783174790901407, "complete-iteration": 0.18287181406813716, "set_robot_commands": 0.002168989692872076, "deviation-center-line": 0.46803497882957223, "driven_lanedir_consec": 0.5180406078051067, "sim_compute_sim_state": 0.008535560909608733, "sim_compute_performance-ego0": 0.002048341582351971}, "LF-norm-zigzag-000-ego0": {"driven_any": 6.296564739392844, "get_ui_image": 0.03401057448216421, "step_physics": 0.12352649992848316, "survival_time": 59.99999999999873, "driven_lanedir": 5.952690535180785, "get_state_dump": 0.00442114062948489, "get_robot_state": 0.0035591324004999904, "sim_render-ego0": 0.003712457582218065, "get_duckie_state": 1.8720126568923685e-06, "in-drivable-lane": 0.9000000000000128, "deviation-heading": 13.362777940549266, "agent_compute-ego0": 0.012133659272269346, "complete-iteration": 0.19689805481852737, "set_robot_commands": 0.002105607875280833, "deviation-center-line": 2.9837212370469395, "driven_lanedir_consec": 5.952690535180785, "sim_compute_sim_state": 0.011419623618717496, "sim_compute_performance-ego0": 0.0019259923304447424}, "LF-norm-techtrack-000-ego0": {"driven_any": 4.883083313052955, "get_ui_image": 0.031747935317283454, "step_physics": 0.12546417502469795, "survival_time": 53.699999999999086, "driven_lanedir": 3.1351823069814695, "get_state_dump": 0.004772864718769872, "get_robot_state": 0.0036142054269480152, "sim_render-ego0": 0.0038959946743277614, "get_duckie_state": 2.1269155103106833e-06, "in-drivable-lane": 15.94999999999942, "deviation-heading": 12.37852608133326, "agent_compute-ego0": 0.01276808073354322, "complete-iteration": 0.197470470251039, "set_robot_commands": 0.0021721664694852607, "deviation-center-line": 1.728179804650386, "driven_lanedir_consec": 3.1351823069814695, "sim_compute_sim_state": 0.010900196696436682, "sim_compute_performance-ego0": 0.002039274393126022}, "LF-norm-small_loop-000-ego0": {"driven_any": 4.93668013398045, "get_ui_image": 0.025906982271002296, "step_physics": 0.10471381890982216, "survival_time": 59.99999999999873, "driven_lanedir": 1.633354821027459, "get_state_dump": 0.004675822293728615, "get_robot_state": 0.00372410495513484, "sim_render-ego0": 0.003900774114832691, "get_duckie_state": 2.10546831802762e-06, "in-drivable-lane": 37.19999999999885, "deviation-heading": 7.517151057405664, "agent_compute-ego0": 0.012724184176010652, "complete-iteration": 0.16606941151678511, "set_robot_commands": 0.002210407828808228, "deviation-center-line": 1.6147440132321669, "driven_lanedir_consec": 1.633354821027459, "sim_compute_sim_state": 0.006126485597481835, "sim_compute_performance-ego0": 0.001999465154668473}}
set_robot_commands_max0.002210407828808228
set_robot_commands_mean0.0021642929666115993
set_robot_commands_median0.002170578081178668
set_robot_commands_min0.002105607875280833
sim_compute_performance-ego0_max0.002048341582351971
sim_compute_performance-ego0_mean0.002003268365147802
sim_compute_performance-ego0_median0.0020193697738972475
sim_compute_performance-ego0_min0.0019259923304447424
sim_compute_sim_state_max0.011419623618717496
sim_compute_sim_state_mean0.009245466705561188
sim_compute_sim_state_median0.009717878803022708
sim_compute_sim_state_min0.006126485597481835
sim_render-ego0_max0.003900774114832691
sim_render-ego0_mean0.003843224150722075
sim_render-ego0_median0.003879832452918773
sim_render-ego0_min0.003712457582218065
simulation-passed1
step_physics_max0.12546417502469795
step_physics_mean0.11814859464540615
step_physics_median0.12120819232355227
step_physics_min0.10471381890982216
survival_time_max59.99999999999873
survival_time_mean48.07499999999917
survival_time_min18.60000000000013
No reset possible
5696811312Moustafa Elarabitemplate-pytorchaido5-LFP-sim-validationLFP-simhost-errornonogpu-prod-040:02:34
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5685711073Melisande Tengexercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-040:23:41
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median3.7046101482252727
survival_time_median45.47499999999952
deviation-center-line_median2.0467817105933737
in-drivable-lane_median14.99999999999951


other stats
agent_compute-ego0_max0.013280780388850355
agent_compute-ego0_mean0.012374875392667689
agent_compute-ego0_median0.012302337504054706
agent_compute-ego0_min0.011614046173710976
complete-iteration_max0.21882140646369927
complete-iteration_mean0.18141308597672692
complete-iteration_median0.17362587486857878
complete-iteration_min0.1595791877060508
deviation-center-line_max2.8503203702911883
deviation-center-line_mean1.7911237651038994
deviation-center-line_min0.220611268937662
deviation-heading_max16.951523277025846
deviation-heading_mean10.192268270417102
deviation-heading_median11.15565174238166
deviation-heading_min1.506246319879238
driven_any_max10.894633376328082
driven_any_mean7.008654685307506
driven_any_median7.807017699841321
driven_any_min1.5259499652193025
driven_lanedir_consec_max6.314579158652464
driven_lanedir_consec_mean3.597717657108941
driven_lanedir_consec_min0.667071173332755
driven_lanedir_max6.668328711269626
driven_lanedir_mean3.927031522619873
driven_lanedir_median4.17236056429863
driven_lanedir_min0.695076250612604
get_duckie_state_max1.60445972364776e-06
get_duckie_state_mean1.384666253616572e-06
get_duckie_state_median1.3435611518396129e-06
get_duckie_state_min1.247082987139302e-06
get_robot_state_max0.003803282367939851
get_robot_state_mean0.0035471933397609964
get_robot_state_median0.0035609341382384797
get_robot_state_min0.0032636227146271737
get_state_dump_max0.004718815258003889
get_state_dump_mean0.0044155002551075715
get_state_dump_median0.004426997824248694
get_state_dump_min0.00408919011392901
get_ui_image_max0.03594682675217907
get_ui_image_mean0.029089482522640386
get_ui_image_median0.02745594722884042
get_ui_image_min0.02549920888070163
in-drivable-lane_max25.599999999999156
in-drivable-lane_mean15.162499999999548
in-drivable-lane_min5.050000000000012
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 10.894633376328082, "get_ui_image": 0.02549920888070163, "step_physics": 0.09836832232320436, "survival_time": 59.99999999999873, "driven_lanedir": 6.668328711269626, "get_state_dump": 0.004176122163555009, "get_robot_state": 0.0033628702362213008, "sim_render-ego0": 0.0035657771521067244, "get_duckie_state": 1.2494642271983633e-06, "in-drivable-lane": 21.549999999998928, "deviation-heading": 11.905099240825722, "agent_compute-ego0": 0.011884493990603533, "complete-iteration": 0.1595791877060508, "set_robot_commands": 0.0019892874010198815, "deviation-center-line": 2.1859952585171594, "driven_lanedir_consec": 6.314579158652464, "sim_compute_sim_state": 0.008826115248503038, "sim_compute_performance-ego0": 0.0018261330610111692}, "LF-norm-zigzag-000-ego0": {"driven_any": 10.402723174692696, "get_ui_image": 0.03594682675217907, "step_physics": 0.14107673610874655, "survival_time": 59.99999999999873, "driven_lanedir": 5.022014577556571, "get_state_dump": 0.004718815258003889, "get_robot_state": 0.0037589980402556586, "sim_render-ego0": 0.004009860838382667, "get_duckie_state": 1.4376580764808625e-06, "in-drivable-lane": 25.599999999999156, "deviation-heading": 16.951523277025846, "agent_compute-ego0": 0.013280780388850355, "complete-iteration": 0.21882140646369927, "set_robot_commands": 0.0022870777647858555, "deviation-center-line": 2.8503203702911883, "driven_lanedir_consec": 4.131514167534067, "sim_compute_sim_state": 0.01156750547201012, "sim_compute_performance-ego0": 0.002082776467468618}, "LF-norm-techtrack-000-ego0": {"driven_any": 5.211312224989946, "get_ui_image": 0.02841758728027344, "step_physics": 0.11425096258040396, "survival_time": 30.950000000000305, "driven_lanedir": 3.3227065510406897, "get_state_dump": 0.00408919011392901, "get_robot_state": 0.0032636227146271737, "sim_render-ego0": 0.0034584087710226737, "get_duckie_state": 1.247082987139302e-06, "in-drivable-lane": 8.450000000000092, "deviation-heading": 10.4062042439376, "agent_compute-ego0": 0.011614046173710976, "complete-iteration": 0.17904838977321502, "set_robot_commands": 0.0018887085299338065, "deviation-center-line": 1.9075681626695875, "driven_lanedir_consec": 3.2777061289164786, "sim_compute_sim_state": 0.010273853425056705, "sim_compute_performance-ego0": 0.0017127610022021878}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.5259499652193025, "get_ui_image": 0.0264943071774074, "step_physics": 0.10648630103286431, "survival_time": 9.750000000000004, "driven_lanedir": 0.695076250612604, "get_state_dump": 0.004677873484942378, "get_robot_state": 0.003803282367939851, "sim_render-ego0": 0.004054619341480489, "get_duckie_state": 1.60445972364776e-06, "in-drivable-lane": 5.050000000000012, "deviation-heading": 1.506246319879238, "agent_compute-ego0": 0.01272018101750588, "complete-iteration": 0.16820335996394253, "set_robot_commands": 0.002348100652500075, "deviation-center-line": 0.220611268937662, "driven_lanedir_consec": 0.667071173332755, "sim_compute_sim_state": 0.005425251260095713, "sim_compute_performance-ego0": 0.0020935924685731226}}
set_robot_commands_max0.002348100652500075
set_robot_commands_mean0.0021282935870599044
set_robot_commands_median0.0021381825829028685
set_robot_commands_min0.0018887085299338065
sim_compute_performance-ego0_max0.0020935924685731226
sim_compute_performance-ego0_mean0.0019288157498137744
sim_compute_performance-ego0_median0.001954454764239893
sim_compute_performance-ego0_min0.0017127610022021878
sim_compute_sim_state_max0.01156750547201012
sim_compute_sim_state_mean0.009023181351416394
sim_compute_sim_state_median0.00954998433677987
sim_compute_sim_state_min0.005425251260095713
sim_render-ego0_max0.004054619341480489
sim_render-ego0_mean0.003772166525748138
sim_render-ego0_median0.003787818995244696
sim_render-ego0_min0.0034584087710226737
simulation-passed1
step_physics_max0.14107673610874655
step_physics_mean0.1150455805113048
step_physics_median0.11036863180663414
step_physics_min0.09836832232320436
survival_time_max59.99999999999873
survival_time_mean40.17499999999944
survival_time_min9.750000000000004
No reset possible
5669611092Melisande Tengexercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-040:26:50
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median3.5163090544043873
survival_time_median49.324999999999335
deviation-center-line_median2.260830047944907
in-drivable-lane_median12.174999999999793


other stats
agent_compute-ego0_max0.012734583797996976
agent_compute-ego0_mean0.012562457661128028
agent_compute-ego0_median0.012609018298916606
agent_compute-ego0_min0.012297210248681923
complete-iteration_max0.21428475946727032
complete-iteration_mean0.17946554407610715
complete-iteration_median0.1732812205990251
complete-iteration_min0.1570149756391082
deviation-center-line_max3.171786801030719
deviation-center-line_mean2.1455793172753093
deviation-center-line_min0.8888703721807046
deviation-heading_max15.175164341108848
deviation-heading_mean10.950989222431032
deviation-heading_median12.39953526808682
deviation-heading_min3.829722012441643
driven_any_max10.649531514443895
driven_any_mean8.117711556231505
driven_any_median8.447861580030835
driven_any_min4.925591550420453
driven_lanedir_consec_max6.909274380617803
driven_lanedir_consec_mean3.994840939613732
driven_lanedir_consec_min2.0374712690283507
driven_lanedir_max8.881616564734662
driven_lanedir_mean5.7177555082304306
driven_lanedir_median5.975967099579355
driven_lanedir_min2.0374712690283507
get_duckie_state_max2.135245825825484e-06
get_duckie_state_mean2.0244445175150736e-06
get_duckie_state_median2.0731150942774736e-06
get_duckie_state_min1.8163020556798645e-06
get_robot_state_max0.003646769988149727
get_robot_state_mean0.003620407356748673
get_robot_state_median0.003633818978896661
get_robot_state_min0.0035672214810516458
get_state_dump_max0.004610590848503803
get_state_dump_mean0.004510509292434127
get_state_dump_median0.004524003853905112
get_state_dump_min0.00438343861342248
get_ui_image_max0.03476066244357008
get_ui_image_mean0.02929463926297901
get_ui_image_median0.028580349821518705
get_ui_image_min0.025257194965308552
in-drivable-lane_max17.65000000000022
in-drivable-lane_mean12.174999999999857
in-drivable-lane_min6.699999999999619
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 10.615705283964576, "get_ui_image": 0.02769648026268647, "step_physics": 0.11094060507940312, "survival_time": 59.99999999999873, "driven_lanedir": 8.881616564734662, "get_state_dump": 0.004587305277015247, "get_robot_state": 0.003634068888490345, "sim_render-ego0": 0.003815587216074719, "get_duckie_state": 2.135245825825484e-06, "in-drivable-lane": 6.699999999999619, "deviation-heading": 15.021287745297885, "agent_compute-ego0": 0.012632582209489427, "complete-iteration": 0.17759125873905535, "set_robot_commands": 0.0021729876258589644, "deviation-center-line": 3.171786801030719, "driven_lanedir_consec": 6.909274380617803, "sim_compute_sim_state": 0.010055828650329234, "sim_compute_performance-ego0": 0.0019731158320850178}, "LF-norm-zigzag-000-ego0": {"driven_any": 6.280017876097097, "get_ui_image": 0.03476066244357008, "step_physics": 0.13934896158617596, "survival_time": 38.64999999999994, "driven_lanedir": 3.447668094325752, "get_state_dump": 0.004610590848503803, "get_robot_state": 0.0036335690693029754, "sim_render-ego0": 0.003858744019993824, "get_duckie_state": 2.035798952561016e-06, "in-drivable-lane": 15.399999999999718, "deviation-heading": 9.777782790875758, "agent_compute-ego0": 0.012734583797996976, "complete-iteration": 0.21428475946727032, "set_robot_commands": 0.00219653003899626, "deviation-center-line": 1.7486493960257792, "driven_lanedir_consec": 2.922854587051115, "sim_compute_sim_state": 0.011039803502479573, "sim_compute_performance-ego0": 0.002020471471840713}, "LF-norm-techtrack-000-ego0": {"driven_any": 4.925591550420453, "get_ui_image": 0.029464219380350945, "step_physics": 0.1008616655004063, "survival_time": 30.850000000000303, "driven_lanedir": 2.0374712690283507, "get_state_dump": 0.00438343861342248, "get_robot_state": 0.0035672214810516458, "sim_render-ego0": 0.003746547745269479, "get_duckie_state": 1.8163020556798645e-06, "in-drivable-lane": 17.65000000000022, "deviation-heading": 3.829722012441643, "agent_compute-ego0": 0.012585454388343789, "complete-iteration": 0.16897118245899484, "set_robot_commands": 0.0021160486060824595, "deviation-center-line": 0.8888703721807046, "driven_lanedir_consec": 2.0374712690283507, "sim_compute_sim_state": 0.01029242435319524, "sim_compute_performance-ego0": 0.001876415558231687}, "LF-norm-small_loop-000-ego0": {"driven_any": 10.649531514443895, "get_ui_image": 0.025257194965308552, "step_physics": 0.09774786189235717, "survival_time": 59.99999999999873, "driven_lanedir": 8.50426610483296, "get_state_dump": 0.004460702430794976, "get_robot_state": 0.003646769988149727, "sim_render-ego0": 0.003707149642194737, "get_duckie_state": 2.110431235993931e-06, "in-drivable-lane": 8.949999999999864, "deviation-heading": 15.175164341108848, "agent_compute-ego0": 0.012297210248681923, "complete-iteration": 0.1570149756391082, "set_robot_commands": 0.002106527206204912, "deviation-center-line": 2.7730106998640345, "driven_lanedir_consec": 4.109763521757659, "sim_compute_sim_state": 0.005820575105856102, "sim_compute_performance-ego0": 0.001883888721069031}}
set_robot_commands_max0.00219653003899626
set_robot_commands_mean0.0021480233692856487
set_robot_commands_median0.002144518115970712
set_robot_commands_min0.002106527206204912
sim_compute_performance-ego0_max0.002020471471840713
sim_compute_performance-ego0_mean0.001938472895806612
sim_compute_performance-ego0_median0.0019285022765770244
sim_compute_performance-ego0_min0.001876415558231687
sim_compute_sim_state_max0.011039803502479573
sim_compute_sim_state_mean0.009302157902965038
sim_compute_sim_state_median0.010174126501762238
sim_compute_sim_state_min0.005820575105856102
sim_render-ego0_max0.003858744019993824
sim_render-ego0_mean0.0037820071558831896
sim_render-ego0_median0.0037810674806720993
sim_render-ego0_min0.003707149642194737
simulation-passed1
step_physics_max0.13934896158617596
step_physics_mean0.11222477351458564
step_physics_median0.10590113528990472
step_physics_min0.09774786189235717
survival_time_max59.99999999999873
survival_time_mean47.374999999999424
survival_time_min30.850000000000303
No reset possible
5668911290Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:00:41
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5668711290Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:00:49
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5665611278Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:03:41
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 48, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 48, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 48, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 48, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5663411280Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:01:54
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5660811282Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:00:49
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5658911427Frank (Chude) Qian 🇨🇦template-pytorchaido5-LFV_multi-sim-validationLFVmultibodyv-simhost-erroryesnogpu-prod-040:02:00
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5649311290Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simtimeoutyesnogpu-prod-040:08:44
Timeout because eval [...]
Timeout because evaluator contacted us
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5645711281Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:01:51
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5641811284Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:02:28
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5637711285Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:01:06
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5632311287Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:03:03
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5630311283Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:00:42
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5623211425Frank (Chude) Qian 🇨🇦template-pytorchaido5-LFP-sim-validationLFP-simhost-errornonogpu-prod-040:03:40
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5616911286Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:02:43
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5611811310Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:01:14
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5610911310Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:00:43
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5606911296Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:01:57
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5606111296Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:00:43
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5597411303Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:05:08
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5595511299Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:01:00
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5593611305Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:00:47
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5590411289Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:00:42
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5589111311Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:00:42
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5585411311Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:02:13
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5578511308Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:03:35
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5570911157Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-040:03:26
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.48910950271291087
survival_time_median4.049999999999994
deviation-center-line_median0.09524551841759736
in-drivable-lane_median2.5249999999999932


other stats
agent_compute-ego0_max0.022233865477822044
agent_compute-ego0_mean0.020097169253470732
agent_compute-ego0_median0.01980676876532065
agent_compute-ego0_min0.01854127400541959
complete-iteration_max0.19768232912630648
complete-iteration_mean0.1737980038814679
complete-iteration_median0.1724844576165408
complete-iteration_min0.15254077116648357
deviation-center-line_max0.12743012060774722
deviation-center-line_mean0.09319976500674051
deviation-center-line_min0.05487790258402013
deviation-heading_max0.971777458646476
deviation-heading_mean0.6054812802598228
deviation-heading_median0.5906084931598334
deviation-heading_min0.26893067607314874
driven_any_max3.037735797168683
driven_any_mean1.7658980823893962
driven_any_median1.4723969329865807
driven_any_min1.0810626664157394
driven_lanedir_consec_max0.6538396753236417
driven_lanedir_consec_mean0.4782744852225676
driven_lanedir_consec_min0.28103926014080693
driven_lanedir_max0.6538396753236417
driven_lanedir_mean0.4782744852225676
driven_lanedir_median0.48910950271291087
driven_lanedir_min0.28103926014080693
get_duckie_state_max1.5822323885830969e-06
get_duckie_state_mean1.438599413801217e-06
get_duckie_state_median1.4675188753084975e-06
get_duckie_state_min1.2371275160047744e-06
get_robot_state_max0.003587018953610773
get_robot_state_mean0.003503118014408656
get_robot_state_median0.003552717326802372
get_robot_state_min0.003320018450419108
get_state_dump_max0.004756411990603885
get_state_dump_mean0.00458299896566022
get_state_dump_median0.0046611458941483806
get_state_dump_min0.004253292083740234
get_ui_image_max0.03176961396191571
get_ui_image_mean0.027609458285107684
get_ui_image_median0.027225514747830235
get_ui_image_min0.024217189682854547
in-drivable-lane_max5.999999999999984
in-drivable-lane_mean3.1374999999999917
in-drivable-lane_min1.4999999999999956
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 3.037735797168683, "get_ui_image": 0.025247871059260955, "step_physics": 0.08602095793371331, "survival_time": 7.249999999999982, "driven_lanedir": 0.28103926014080693, "get_state_dump": 0.004599530402928182, "get_robot_state": 0.003587018953610773, "sim_render-ego0": 0.003787476722508261, "get_duckie_state": 1.54318874829436e-06, "in-drivable-lane": 5.999999999999984, "deviation-heading": 0.8792237769857258, "agent_compute-ego0": 0.01854127400541959, "complete-iteration": 0.15498067418189898, "set_robot_commands": 0.002174323552275357, "deviation-center-line": 0.12743012060774722, "driven_lanedir_consec": 0.28103926014080693, "sim_compute_sim_state": 0.008925594695626872, "sim_compute_performance-ego0": 0.0020080151623242523}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.2767296026240689, "get_ui_image": 0.03176961396191571, "step_physics": 0.12029830185142724, "survival_time": 3.649999999999995, "driven_lanedir": 0.3560000663109024, "get_state_dump": 0.004756411990603885, "get_robot_state": 0.0035289139360994907, "sim_render-ego0": 0.0037161466237661, "get_duckie_state": 1.3918490023226351e-06, "in-drivable-lane": 2.2499999999999947, "deviation-heading": 0.971777458646476, "agent_compute-ego0": 0.02076445076916669, "complete-iteration": 0.19768232912630648, "set_robot_commands": 0.0021363818967664564, "deviation-center-line": 0.10079821615328478, "driven_lanedir_consec": 0.3560000663109024, "sim_compute_sim_state": 0.008694316889788653, "sim_compute_performance-ego0": 0.001932801427067937}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.0810626664157394, "get_ui_image": 0.029203158436399517, "step_physics": 0.11506765177755644, "survival_time": 3.2499999999999964, "driven_lanedir": 0.6538396753236417, "get_state_dump": 0.004722761385368578, "get_robot_state": 0.003576520717505253, "sim_render-ego0": 0.0037464336915449662, "get_duckie_state": 1.5822323885830969e-06, "in-drivable-lane": 1.4999999999999956, "deviation-heading": 0.30199320933394114, "agent_compute-ego0": 0.022233865477822044, "complete-iteration": 0.18998824105118264, "set_robot_commands": 0.0021463162971265388, "deviation-center-line": 0.05487790258402013, "driven_lanedir_consec": 0.6538396753236417, "sim_compute_sim_state": 0.007275332104076038, "sim_compute_performance-ego0": 0.001925392584367232}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.6680642633490927, "get_ui_image": 0.024217189682854547, "step_physics": 0.0902565982606676, "survival_time": 4.449999999999992, "driven_lanedir": 0.6222189391149193, "get_state_dump": 0.004253292083740234, "get_robot_state": 0.003320018450419108, "sim_render-ego0": 0.0033426390753851995, "get_duckie_state": 1.2371275160047744e-06, "in-drivable-lane": 2.7999999999999914, "deviation-heading": 0.26893067607314874, "agent_compute-ego0": 0.01884908676147461, "complete-iteration": 0.15254077116648357, "set_robot_commands": 0.001968166563245986, "deviation-center-line": 0.08969282068190995, "driven_lanedir_consec": 0.6222189391149193, "sim_compute_sim_state": 0.004547929763793946, "sim_compute_performance-ego0": 0.0017068227132161458}}
set_robot_commands_max0.002174323552275357
set_robot_commands_mean0.002106297077353584
set_robot_commands_median0.0021413490969464976
set_robot_commands_min0.001968166563245986
sim_compute_performance-ego0_max0.0020080151623242523
sim_compute_performance-ego0_mean0.001893257971743892
sim_compute_performance-ego0_median0.0019290970057175845
sim_compute_performance-ego0_min0.0017068227132161458
sim_compute_sim_state_max0.008925594695626872
sim_compute_sim_state_mean0.007360793363321377
sim_compute_sim_state_median0.007984824496932346
sim_compute_sim_state_min0.004547929763793946
sim_render-ego0_max0.003787476722508261
sim_render-ego0_mean0.003648174028301131
sim_render-ego0_median0.003731290157655533
sim_render-ego0_min0.0033426390753851995
simulation-passed1
step_physics_max0.12029830185142724
step_physics_mean0.10291087745584113
step_physics_median0.102662125019112
step_physics_min0.08602095793371331
survival_time_max7.249999999999982
survival_time_mean4.6499999999999915
survival_time_min3.2499999999999964
No reset possible
5567211302Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:00:50
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5540911219Charlie Gauthier 🇨🇦Baseline solution using reinforcement learningaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-040:10:51
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.4946117585028367
survival_time_median3.724999999999995
deviation-center-line_median0.09976542858101645
in-drivable-lane_median2.5999999999999948


other stats
agent_compute-ego0_max0.05652210810413099
agent_compute-ego0_mean0.027762314970616687
agent_compute-ego0_median0.020880829192695717
agent_compute-ego0_min0.012765493392944336
complete-iteration_max0.20651023011458547
complete-iteration_mean0.1969958481480491
complete-iteration_median0.19872191980004855
complete-iteration_min0.18402932287751383
deviation-center-line_max1.371790606877935
deviation-center-line_mean0.4055973938957933
deviation-center-line_min0.05106811154320492
deviation-heading_max6.413654703558596
deviation-heading_mean2.013607641383015
deviation-heading_median0.6953287013797487
deviation-heading_min0.2501184592139676
driven_any_max31.926895416617864
driven_any_mean9.04642867729396
driven_any_median1.44108474357067
driven_any_min1.3766498054166303
driven_lanedir_consec_max6.525550803294335
driven_lanedir_consec_mean1.9625023349278163
driven_lanedir_consec_min0.335235019411257
driven_lanedir_max6.525550803294335
driven_lanedir_mean1.966200964039724
driven_lanedir_median0.5020090167266513
driven_lanedir_min0.335235019411257
get_duckie_state_max1.0813205664997594e-06
get_duckie_state_mean1.056022640103168e-06
get_duckie_state_median1.0553342566642583e-06
get_duckie_state_min1.0321014805843957e-06
get_robot_state_max0.0033287922607472694
get_robot_state_mean0.0032495630949955826
get_robot_state_median0.0032516495169025578
get_robot_state_min0.0031661610854299446
get_state_dump_max0.004183147471711399
get_state_dump_mean0.004039666939638914
get_state_dump_median0.004052071070562215
get_state_dump_min0.0038713781457198294
get_ui_image_max0.029805616328590794
get_ui_image_mean0.02651580428173117
get_ui_image_median0.02628170118641595
get_ui_image_min0.02369419842550199
in-drivable-lane_max45.69999999999902
in-drivable-lane_mean13.174999999999752
in-drivable-lane_min1.7999999999999945
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 31.926895416617864, "get_ui_image": 0.024770595747465693, "step_physics": 0.10940029877210834, "survival_time": 59.99999999999873, "driven_lanedir": 6.525550803294335, "get_state_dump": 0.004183147471711399, "get_robot_state": 0.0033287922607472694, "sim_render-ego0": 0.0034315143397805298, "get_duckie_state": 1.0813205664997594e-06, "in-drivable-lane": 45.69999999999902, "deviation-heading": 6.413654703558596, "agent_compute-ego0": 0.02787393614414034, "complete-iteration": 0.18402932287751383, "set_robot_commands": 0.002169979700537943, "deviation-center-line": 1.371790606877935, "driven_lanedir_consec": 6.525550803294335, "sim_compute_sim_state": 0.007001566747940152, "sim_compute_performance-ego0": 0.0017888448716798094}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.4519713763589728, "get_ui_image": 0.029805616328590794, "step_physics": 0.14064065719905652, "survival_time": 3.7499999999999942, "driven_lanedir": 0.6186942718785522, "get_state_dump": 0.0038713781457198294, "get_robot_state": 0.0031661610854299446, "sim_render-ego0": 0.003305513607828241, "get_duckie_state": 1.0321014805843957e-06, "in-drivable-lane": 1.7999999999999945, "deviation-heading": 1.1322069560818493, "agent_compute-ego0": 0.013887722241251092, "complete-iteration": 0.20651023011458547, "set_robot_commands": 0.002259944614611174, "deviation-center-line": 0.145486644509924, "driven_lanedir_consec": 0.6038997554309229, "sim_compute_sim_state": 0.007781088352203369, "sim_compute_performance-ego0": 0.0017159487071790192}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.4301981107823671, "get_ui_image": 0.02779280662536621, "step_physics": 0.1310219446818034, "survival_time": 3.699999999999995, "driven_lanedir": 0.38532376157475046, "get_state_dump": 0.004044984181722005, "get_robot_state": 0.0032981204986572265, "sim_render-ego0": 0.003284193674723307, "get_duckie_state": 1.0426839192708333e-06, "in-drivable-lane": 2.5999999999999943, "deviation-heading": 0.25845044667764816, "agent_compute-ego0": 0.012765493392944336, "complete-iteration": 0.1926813252766927, "set_robot_commands": 0.0021387863159179686, "deviation-center-line": 0.05106811154320492, "driven_lanedir_consec": 0.38532376157475046, "sim_compute_sim_state": 0.006549886067708333, "sim_compute_performance-ego0": 0.001706062952677409}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.3766498054166303, "get_ui_image": 0.02369419842550199, "step_physics": 0.10566492276648952, "survival_time": 3.599999999999995, "driven_lanedir": 0.335235019411257, "get_state_dump": 0.004059157959402424, "get_robot_state": 0.003205178535147889, "sim_render-ego0": 0.0032662006273661576, "get_duckie_state": 1.067984594057684e-06, "in-drivable-lane": 2.599999999999995, "deviation-heading": 0.2501184592139676, "agent_compute-ego0": 0.05652210810413099, "complete-iteration": 0.2047625143234044, "set_robot_commands": 0.0021775977252280875, "deviation-center-line": 0.05404421265210891, "driven_lanedir_consec": 0.335235019411257, "sim_compute_sim_state": 0.004402977146514475, "sim_compute_performance-ego0": 0.0016907796467820258}}
set_robot_commands_max0.002259944614611174
set_robot_commands_mean0.002186577089073794
set_robot_commands_median0.002173788712883015
set_robot_commands_min0.0021387863159179686
sim_compute_performance-ego0_max0.0017888448716798094
sim_compute_performance-ego0_mean0.001725409044579566
sim_compute_performance-ego0_median0.001711005829928214
sim_compute_performance-ego0_min0.0016907796467820258
sim_compute_sim_state_max0.007781088352203369
sim_compute_sim_state_mean0.006433879578591583
sim_compute_sim_state_median0.006775726407824242
sim_compute_sim_state_min0.004402977146514475
sim_render-ego0_max0.0034315143397805298
sim_render-ego0_mean0.0033218555624245586
sim_render-ego0_median0.003294853641275774
sim_render-ego0_min0.0032662006273661576
simulation-passed1
step_physics_max0.14064065719905652
step_physics_mean0.12168195585486444
step_physics_median0.12021112172695586
step_physics_min0.10566492276648952
survival_time_max59.99999999999873
survival_time_mean17.76249999999968
survival_time_min3.599999999999995
No reset possible
5535611301Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:00:47
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5514411413Frank (Chude) Qian 🇨🇦challenge-aido_LF-baseline-behavior-cloningaido5-LFP-sim-validationLFP-simsuccessnonogpu-prod-040:12:45
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median8.025
in-drivable-lane_median0.5250000000000075
driven_lanedir_consec_median1.5425060846496597
deviation-center-line_median0.3402907324174555


other stats
agent_compute-ego0_max0.04811408886542687
agent_compute-ego0_mean0.046050046983499016
agent_compute-ego0_median0.046425945065093274
agent_compute-ego0_min0.043234208938382646
complete-iteration_max0.30368735973651595
complete-iteration_mean0.24968040867171987
complete-iteration_median0.24943260343676824
complete-iteration_min0.19616906807682696
deviation-center-line_max3.9835052701742866
deviation-center-line_mean1.2333460279122046
deviation-center-line_min0.2692973766396206
deviation-heading_max15.44160972654172
deviation-heading_mean4.827289859714541
deviation-heading_median1.5082645857802492
deviation-heading_min0.851020540755943
driven_any_max14.5898100863319
driven_any_mean4.633620375837103
driven_any_median1.7082199312507167
driven_any_min0.528231554515081
driven_lanedir_consec_max11.088425839240465
driven_lanedir_consec_mean3.66648067402276
driven_lanedir_consec_min0.49248468755125585
driven_lanedir_max11.088425839240465
driven_lanedir_mean3.66648067402276
driven_lanedir_median1.5425060846496597
driven_lanedir_min0.49248468755125585
get_duckie_state_max0.024166626425889823
get_duckie_state_mean0.017143787061329258
get_duckie_state_median0.020167820031028925
get_duckie_state_min0.004072881757369347
get_robot_state_max0.003766569724449745
get_robot_state_mean0.003557820855677623
get_robot_state_median0.003515192643662155
get_robot_state_min0.0034343284109364385
get_state_dump_max0.008195347510851346
get_state_dump_mean0.007141564647862532
get_state_dump_median0.007663025584906638
get_state_dump_min0.0050448599107855065
get_ui_image_max0.036608255826509915
get_ui_image_mean0.03111001453126799
get_ui_image_median0.03150771568750458
get_ui_image_min0.02481637092355288
in-drivable-lane_max11.099999999999756
in-drivable-lane_mean3.037499999999943
in-drivable-lane_min0.0
per-episodes
details{"LFP-norm-loop-000-ego0": {"driven_any": 2.276806063496427, "get_ui_image": 0.030151198689754192, "step_physics": 0.1117120488331868, "survival_time": 10.350000000000012, "driven_lanedir": 1.9665277864594513, "get_state_dump": 0.008195347510851346, "get_robot_state": 0.003542355619944059, "sim_render-ego0": 0.0035657699291522685, "get_duckie_state": 0.024166626425889823, "in-drivable-lane": 1.050000000000015, "deviation-heading": 1.747008755262681, "agent_compute-ego0": 0.04811408886542687, "complete-iteration": 0.2410441258778939, "set_robot_commands": 0.0021957743626374463, "deviation-center-line": 0.3724996356547582, "driven_lanedir_consec": 1.9665277864594513, "sim_compute_sim_state": 0.007503641339448782, "sim_compute_performance-ego0": 0.0018092840909957888}, "LFP-norm-zigzag-000-ego0": {"driven_any": 0.528231554515081, "get_ui_image": 0.036608255826509915, "step_physics": 0.17042846312889687, "survival_time": 3.1999999999999966, "driven_lanedir": 0.49248468755125585, "get_state_dump": 0.007995898907001201, "get_robot_state": 0.003766569724449745, "sim_render-ego0": 0.003691291809082031, "get_duckie_state": 0.02074722143319937, "in-drivable-lane": 0.0, "deviation-heading": 1.2695204162978178, "agent_compute-ego0": 0.04686522117027869, "complete-iteration": 0.30368735973651595, "set_robot_commands": 0.0022930255303016075, "deviation-center-line": 0.2692973766396206, "driven_lanedir_consec": 0.49248468755125585, "sim_compute_sim_state": 0.009194462115948016, "sim_compute_performance-ego0": 0.0019995579352745643}, "LFP-norm-techtrack-000-ego0": {"driven_any": 1.1396337990050067, "get_ui_image": 0.03286423268525497, "step_physics": 0.12901154186414637, "survival_time": 5.699999999999988, "driven_lanedir": 1.118484382839868, "get_state_dump": 0.007330152262812075, "get_robot_state": 0.0034343284109364385, "sim_render-ego0": 0.00354955300040867, "get_duckie_state": 0.01958841862885848, "in-drivable-lane": 0.0, "deviation-heading": 0.851020540755943, "agent_compute-ego0": 0.04598666895990786, "complete-iteration": 0.2578210809956426, "set_robot_commands": 0.002118805180425229, "deviation-center-line": 0.3080818291801528, "driven_lanedir_consec": 1.118484382839868, "sim_compute_sim_state": 0.01202053401781165, "sim_compute_performance-ego0": 0.001828674648119056}, "LFP-norm-small_loop-000-ego0": {"driven_any": 14.5898100863319, "get_ui_image": 0.02481637092355288, "step_physics": 0.10205903617071171, "survival_time": 59.99999999999873, "driven_lanedir": 11.088425839240465, "get_state_dump": 0.0050448599107855065, "get_robot_state": 0.003488029667380251, "sim_render-ego0": 0.003540605827731753, "get_duckie_state": 0.004072881757369347, "in-drivable-lane": 11.099999999999756, "deviation-heading": 15.44160972654172, "agent_compute-ego0": 0.043234208938382646, "complete-iteration": 0.19616906807682696, "set_robot_commands": 0.0021299091008779507, "deviation-center-line": 3.9835052701742866, "driven_lanedir_consec": 11.088425839240465, "sim_compute_sim_state": 0.005880607157126751, "sim_compute_performance-ego0": 0.0018190573296082409}}
set_robot_commands_max0.0022930255303016075
set_robot_commands_mean0.0021843785435605584
set_robot_commands_median0.0021628417317576983
set_robot_commands_min0.002118805180425229
sim_compute_performance-ego0_max0.0019995579352745643
sim_compute_performance-ego0_mean0.0018641435009994125
sim_compute_performance-ego0_median0.001823865988863648
sim_compute_performance-ego0_min0.0018092840909957888
sim_compute_sim_state_max0.01202053401781165
sim_compute_sim_state_mean0.0086498111575838
sim_compute_sim_state_median0.0083490517276984
sim_compute_sim_state_min0.005880607157126751
sim_render-ego0_max0.003691291809082031
sim_render-ego0_mean0.003586805141593681
sim_render-ego0_median0.0035576614647804695
sim_render-ego0_min0.003540605827731753
simulation-passed1
step_physics_max0.17042846312889687
step_physics_mean0.12830277249923544
step_physics_median0.12036179534866658
step_physics_min0.10205903617071171
survival_time_max59.99999999999873
survival_time_mean19.81249999999968
survival_time_min3.1999999999999966
No reset possible
5510411291Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:02:08
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5506611291Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:01:00
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5503411450Cliff Chandler 🇺🇸template-pytorchaido5-LF-sim-testingLFt-simhost-erroryesnogpu-prod-040:02:53
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5471211468Raphael Jeanexercise_state_estimationaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-040:42:43
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.0
survival_time_median59.99999999999873
deviation-center-line_median1.2422730096440104
in-drivable-lane_median0.0


other stats
agent_compute-ego0_max0.012184464663490466
agent_compute-ego0_mean0.011968813172784276
agent_compute-ego0_median0.012032863797990604
agent_compute-ego0_min0.011625060431665423
complete-iteration_max0.3546252957390905
complete-iteration_mean0.29263127902266783
complete-iteration_median0.2859390395368565
complete-iteration_min0.2440217412778678
deviation-center-line_max4.053503393024394
deviation-center-line_mean1.731069028434027
deviation-center-line_min0.386226701423694
deviation-heading_max27.859809596736422
deviation-heading_mean14.925250437835436
deviation-heading_median14.309269950178932
deviation-heading_min3.22265225424745
driven_any_max2.6645352591003757e-13
driven_any_mean1.9984014443252818e-13
driven_any_median2.6645352591003757e-13
driven_any_min0.0
driven_lanedir_consec_max0.000286102294921875
driven_lanedir_consec_mean7.152557373046875e-05
driven_lanedir_consec_min0.0
driven_lanedir_max0.000286102294921875
driven_lanedir_mean7.152557373046875e-05
driven_lanedir_median0.0
driven_lanedir_min0.0
get_duckie_state_max1.3012770907666462e-06
get_duckie_state_mean1.2836091028065804e-06
get_duckie_state_median1.2931379053018969e-06
get_duckie_state_min1.2468835098558818e-06
get_robot_state_max0.0037630809336081823
get_robot_state_mean0.003667061821209402
get_robot_state_median0.003672553935118461
get_robot_state_min0.0035600584809925037
get_state_dump_max0.00504055785497559
get_state_dump_mean0.004704055291826183
get_state_dump_median0.004645289529074639
get_state_dump_min0.004485084254179867
get_ui_image_max0.03574515560286726
get_ui_image_mean0.030338568949481032
get_ui_image_median0.02949863200779263
get_ui_image_min0.026611856179471614
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 2.6645352591003757e-13, "get_ui_image": 0.027733984239691003, "step_physics": 0.20959883466747575, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "get_state_dump": 0.004526903587614468, "get_robot_state": 0.0035600584809925037, "sim_render-ego0": 0.003684714870786389, "get_duckie_state": 1.2959171393630309e-06, "in-drivable-lane": 0.0, "deviation-heading": 22.66279310353771, "agent_compute-ego0": 0.011625060431665423, "complete-iteration": 0.27337985193600367, "set_robot_commands": 0.0021393987955796927, "deviation-center-line": 4.053503393024394, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.008488998722772019, "sim_compute_performance-ego0": 0.0019369167054721856}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.0, "get_ui_image": 0.03574515560286726, "step_physics": 0.27800794465654993, "survival_time": 59.99999999999873, "driven_lanedir": 0.000286102294921875, "get_state_dump": 0.004763675470534808, "get_robot_state": 0.0037630809336081823, "sim_render-ego0": 0.0038879318697863474, "get_duckie_state": 1.3012770907666462e-06, "in-drivable-lane": 0.0, "deviation-heading": 27.859809596736422, "agent_compute-ego0": 0.012184464663490466, "complete-iteration": 0.3546252957390905, "set_robot_commands": 0.0022367265798170105, "deviation-center-line": 1.0457540566888746, "driven_lanedir_consec": 0.000286102294921875, "sim_compute_sim_state": 0.011871808970798365, "sim_compute_performance-ego0": 0.0020699199292185306}, "LF-norm-techtrack-000-ego0": {"driven_any": 2.6645352591003757e-13, "get_ui_image": 0.03126327977589426, "step_physics": 0.2302491073306653, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "get_state_dump": 0.004485084254179867, "get_robot_state": 0.003664477480936805, "sim_render-ego0": 0.00377945975399732, "get_duckie_state": 1.2468835098558818e-06, "in-drivable-lane": 0.0, "deviation-heading": 3.22265225424745, "agent_compute-ego0": 0.01217063717996945, "complete-iteration": 0.29849822713770935, "set_robot_commands": 0.002174897158175682, "deviation-center-line": 1.4387919625991463, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.008654466775136625, "sim_compute_performance-ego0": 0.0019748214877316796}, "LF-norm-small_loop-000-ego0": {"driven_any": 2.6645352591003757e-13, "get_ui_image": 0.026611856179471614, "step_physics": 0.18232923066189247, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "get_state_dump": 0.00504055785497559, "get_robot_state": 0.003680630389300115, "sim_render-ego0": 0.003919612358849214, "get_duckie_state": 1.2903586712407628e-06, "in-drivable-lane": 0.0, "deviation-heading": 5.955746796820156, "agent_compute-ego0": 0.011895090416011762, "complete-iteration": 0.2440217412778678, "set_robot_commands": 0.002222992200637042, "deviation-center-line": 0.386226701423694, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.0062505073293261885, "sim_compute_performance-ego0": 0.001985452455048954}}
set_robot_commands_max0.0022367265798170105
set_robot_commands_mean0.0021935036835523565
set_robot_commands_median0.0021989446794063623
set_robot_commands_min0.0021393987955796927
sim_compute_performance-ego0_max0.0020699199292185306
sim_compute_performance-ego0_mean0.0019917776443678374
sim_compute_performance-ego0_median0.0019801369713903167
sim_compute_performance-ego0_min0.0019369167054721856
sim_compute_sim_state_max0.011871808970798365
sim_compute_sim_state_mean0.008816445449508298
sim_compute_sim_state_median0.00857173274895432
sim_compute_sim_state_min0.0062505073293261885
sim_render-ego0_max0.003919612358849214
sim_render-ego0_mean0.0038179297133548178
sim_render-ego0_median0.003833695811891834
sim_render-ego0_min0.003684714870786389
simulation-passed1
step_physics_max0.27800794465654993
step_physics_mean0.2250462793291459
step_physics_median0.21992397099907057
step_physics_min0.18232923066189247
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
5467611449Cliff Chandler 🇺🇸template-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:02:14
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5463811436Charlie Gauthier 🇨🇦baseline-RL-sim-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:02:06
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 23, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 43, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 23, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 43, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 23, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 43, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 23, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 43, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5461111424Frank (Chude) Qian 🇨🇦template-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:02:03
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5459311438Charlie Gauthier 🇨🇦baseline-RL-sim-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:02:30
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 23, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 43, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 23, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 43, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 23, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 43, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 23, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 43, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5454111462Jerome Labonte 🇨🇦sim-exercise-2aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-040:18:40
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.0046808156827898
survival_time_median19.325000000000134
deviation-center-line_median0.673371464595942
in-drivable-lane_median6.07499999999999


other stats
agent_compute-ego0_max0.014228936397668089
agent_compute-ego0_mean0.013055016743688572
agent_compute-ego0_median0.012886805667100728
agent_compute-ego0_min0.012217519242884748
complete-iteration_max0.2388292969884099
complete-iteration_mean0.21074747830277415
complete-iteration_median0.21006026299843497
complete-iteration_min0.18404009022581683
deviation-center-line_max3.5351345602867665
deviation-center-line_mean1.255257596345915
deviation-center-line_min0.13915289590500954
deviation-heading_max11.999199871862476
deviation-heading_mean4.907107083472715
deviation-heading_median3.558738548811905
deviation-heading_min0.511751364404573
driven_any_max15.247548659818738
driven_any_mean5.937798096929549
driven_any_median3.467789741705503
driven_any_min1.5680642444884494
driven_lanedir_consec_max14.463808611785636
driven_lanedir_consec_mean4.2479267691057325
driven_lanedir_consec_min0.5185368332717115
driven_lanedir_max14.463808611785636
driven_lanedir_mean4.620595603262206
driven_lanedir_median1.7500184839957384
driven_lanedir_min0.5185368332717115
get_duckie_state_max1.6487005985144413e-06
get_duckie_state_mean1.442030791075961e-06
get_duckie_state_median1.3916032662699652e-06
get_duckie_state_min1.336216033249473e-06
get_robot_state_max0.004085894786950314
get_robot_state_mean0.00390710132409818
get_robot_state_median0.0038777701362141545
get_robot_state_min0.0037869702370140978
get_state_dump_max0.006160569913459547
get_state_dump_mean0.00510196251887615
get_state_dump_median0.004796555943801014
get_state_dump_min0.004654168274443028
get_ui_image_max0.035797180159617276
get_ui_image_mean0.03285591907598782
get_ui_image_median0.03310157217811503
get_ui_image_min0.02942335178810393
in-drivable-lane_max12.600000000000154
in-drivable-lane_mean6.625000000000028
in-drivable-lane_min1.7499999999999774
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 15.247548659818738, "get_ui_image": 0.02942335178810393, "step_physics": 0.11548880573911136, "survival_time": 59.99999999999873, "driven_lanedir": 14.463808611785636, "get_state_dump": 0.00466245715564534, "get_robot_state": 0.0037869702370140978, "sim_render-ego0": 0.0038917346560488537, "get_duckie_state": 1.336216033249473e-06, "in-drivable-lane": 1.7499999999999774, "deviation-heading": 11.999199871862476, "agent_compute-ego0": 0.012572485441768496, "complete-iteration": 0.18404009022581683, "set_robot_commands": 0.0023653066525550605, "deviation-center-line": 3.5351345602867665, "driven_lanedir_consec": 14.463808611785636, "sim_compute_sim_state": 0.009634904917034877, "sim_compute_performance-ego0": 0.0021220829762785163}, "LF-norm-zigzag-000-ego0": {"driven_any": 5.334052611441604, "get_ui_image": 0.035797180159617276, "step_physics": 0.1470478938797773, "survival_time": 29.450000000000284, "driven_lanedir": 2.853343159377217, "get_state_dump": 0.004654168274443028, "get_robot_state": 0.003831482337693037, "sim_render-ego0": 0.003909087585190595, "get_duckie_state": 1.3565613051592293e-06, "in-drivable-lane": 12.600000000000154, "deviation-heading": 6.584815901815964, "agent_compute-ego0": 0.012217519242884748, "complete-iteration": 0.2234980098271774, "set_robot_commands": 0.0022698572126485534, "deviation-center-line": 1.0992854413088355, "driven_lanedir_consec": 1.36266782275132, "sim_compute_sim_state": 0.01160070855738753, "sim_compute_performance-ego0": 0.002081240233728441}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.6015268719694018, "get_ui_image": 0.03570313840299039, "step_physics": 0.16318064251461545, "survival_time": 9.199999999999996, "driven_lanedir": 0.5185368332717115, "get_state_dump": 0.004930654731956688, "get_robot_state": 0.003924057934735273, "sim_render-ego0": 0.004093190786000844, "get_duckie_state": 1.426645227380701e-06, "in-drivable-lane": 6.899999999999996, "deviation-heading": 0.511751364404573, "agent_compute-ego0": 0.01320112589243296, "complete-iteration": 0.2388292969884099, "set_robot_commands": 0.002451116974289353, "deviation-center-line": 0.13915289590500954, "driven_lanedir_consec": 0.5185368332717115, "sim_compute_sim_state": 0.009077933027937603, "sim_compute_performance-ego0": 0.002171171033704603}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.5680642444884494, "get_ui_image": 0.03050000595323967, "step_physics": 0.12676368626681242, "survival_time": 8.199999999999982, "driven_lanedir": 0.6466938086142597, "get_state_dump": 0.006160569913459547, "get_robot_state": 0.004085894786950314, "sim_render-ego0": 0.004068189678770123, "get_duckie_state": 1.6487005985144413e-06, "in-drivable-lane": 5.249999999999984, "deviation-heading": 0.5326611958078469, "agent_compute-ego0": 0.014228936397668089, "complete-iteration": 0.1966225161696925, "set_robot_commands": 0.002562956376509233, "deviation-center-line": 0.2474574878830484, "driven_lanedir_consec": 0.6466938086142597, "sim_compute_sim_state": 0.005937317645911014, "sim_compute_performance-ego0": 0.0022093238252581973}}
set_robot_commands_max0.002562956376509233
set_robot_commands_mean0.00241230930400055
set_robot_commands_median0.0024082118134222067
set_robot_commands_min0.0022698572126485534
sim_compute_performance-ego0_max0.0022093238252581973
sim_compute_performance-ego0_mean0.0021459545172424392
sim_compute_performance-ego0_median0.0021466270049915596
sim_compute_performance-ego0_min0.002081240233728441
sim_compute_sim_state_max0.01160070855738753
sim_compute_sim_state_mean0.009062716037067755
sim_compute_sim_state_median0.00935641897248624
sim_compute_sim_state_min0.005937317645911014
sim_render-ego0_max0.004093190786000844
sim_render-ego0_mean0.003990550676502604
sim_render-ego0_median0.00398863863198036
sim_render-ego0_min0.0038917346560488537
simulation-passed1
step_physics_max0.16318064251461545
step_physics_mean0.13812025710007914
step_physics_median0.13690579007329484
step_physics_min0.11548880573911136
survival_time_max59.99999999999873
survival_time_mean26.71249999999975
survival_time_min8.199999999999982
No reset possible
5449311500Olga (Ge Ya) Xu 🇨🇦sim-exercise-2aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-040:14:39
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.6977548702811781
survival_time_median14.225000000000067
deviation-center-line_median0.3202941083267172
in-drivable-lane_median10.550000000000075


other stats
agent_compute-ego0_max0.013078180806977409
agent_compute-ego0_mean0.012700155110587544
agent_compute-ego0_median0.012707297003097109
agent_compute-ego0_min0.01230784562917856
complete-iteration_max0.21873705301965987
complete-iteration_mean0.1936895066202372
complete-iteration_median0.1971111694373625
complete-iteration_min0.16179863458656402
deviation-center-line_max2.3470201309425636
deviation-center-line_mean0.7888503134126569
deviation-center-line_min0.16779290605462968
deviation-heading_max13.423404570648495
deviation-heading_mean4.477480586510044
deviation-heading_median1.9105163672945364
deviation-heading_min0.6654850408026043
driven_any_max10.586926029626902
driven_any_mean4.1126849402555345
driven_any_median2.395588279557508
driven_any_min1.07263717228022
driven_lanedir_consec_max4.41910170194589
driven_lanedir_consec_mean1.5177314198690215
driven_lanedir_consec_min0.25631423696783995
driven_lanedir_max4.642517218625121
driven_lanedir_mean1.575989027650509
driven_lanedir_median0.6991693751492586
driven_lanedir_min0.26310014167839846
get_duckie_state_max1.4730862208775112e-06
get_duckie_state_mean1.4453614978605776e-06
get_duckie_state_median1.4521377077833404e-06
get_duckie_state_min1.4040843549981174e-06
get_robot_state_max0.004075322832380023
get_robot_state_mean0.003999622633635266
get_robot_state_median0.0040245948982481775
get_robot_state_min0.003873977905664688
get_state_dump_max0.0051097000369819175
get_state_dump_mean0.0049924288525588145
get_state_dump_median0.005000119926510276
get_state_dump_min0.004859775520232787
get_ui_image_max0.03640987475713094
get_ui_image_mean0.0323337144408872
get_ui_image_median0.03176395795097019
get_ui_image_min0.02939706710447748
in-drivable-lane_max31.94999999999893
in-drivable-lane_mean13.999999999999767
in-drivable-lane_min2.9499999999999895
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 10.586926029626902, "get_ui_image": 0.029813563992439163, "step_physics": 0.1117919656656664, "survival_time": 59.99999999999873, "driven_lanedir": 4.642517218625121, "get_state_dump": 0.0051097000369819175, "get_robot_state": 0.00403658615163125, "sim_render-ego0": 0.004145530538693951, "get_duckie_state": 1.447782429132136e-06, "in-drivable-lane": 31.94999999999893, "deviation-heading": 13.423404570648495, "agent_compute-ego0": 0.012933514100328076, "complete-iteration": 0.18309759776062215, "set_robot_commands": 0.002429047790197012, "deviation-center-line": 2.3470201309425636, "driven_lanedir_consec": 4.41910170194589, "sim_compute_sim_state": 0.010429234627780072, "sim_compute_performance-ego0": 0.002305039954522964}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.2719589227200214, "get_ui_image": 0.03640987475713094, "step_physics": 0.13493953454188812, "survival_time": 7.7499999999999805, "driven_lanedir": 0.26310014167839846, "get_state_dump": 0.005030994231884296, "get_robot_state": 0.003873977905664688, "sim_render-ego0": 0.003891226572868152, "get_duckie_state": 1.4564929864345452e-06, "in-drivable-lane": 4.999999999999987, "deviation-heading": 2.8376868983597263, "agent_compute-ego0": 0.01230784562917856, "complete-iteration": 0.21112474111410287, "set_robot_commands": 0.0022954436448904183, "deviation-center-line": 0.21354814848927753, "driven_lanedir_consec": 0.25631423696783995, "sim_compute_sim_state": 0.010219288177979298, "sim_compute_performance-ego0": 0.0020580337597773625}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.07263717228022, "get_ui_image": 0.033714351909501214, "step_physics": 0.14529307186603546, "survival_time": 5.549999999999988, "driven_lanedir": 0.6454916463506817, "get_state_dump": 0.004969245621136257, "get_robot_state": 0.004075322832380023, "sim_render-ego0": 0.004055257354463849, "get_duckie_state": 1.4730862208775112e-06, "in-drivable-lane": 2.9499999999999895, "deviation-heading": 0.9833458362293466, "agent_compute-ego0": 0.013078180806977409, "complete-iteration": 0.21873705301965987, "set_robot_commands": 0.002351894974708557, "deviation-center-line": 0.16779290605462968, "driven_lanedir_consec": 0.6426626366145207, "sim_compute_sim_state": 0.008931170616831099, "sim_compute_performance-ego0": 0.002170317939349583}, "LF-norm-small_loop-000-ego0": {"driven_any": 3.519217636394994, "get_ui_image": 0.02939706710447748, "step_physics": 0.09599982456988598, "survival_time": 20.70000000000016, "driven_lanedir": 0.7528471039478355, "get_state_dump": 0.004859775520232787, "get_robot_state": 0.004012603644865105, "sim_render-ego0": 0.0039856973900852435, "get_duckie_state": 1.4040843549981174e-06, "in-drivable-lane": 16.10000000000017, "deviation-heading": 0.6654850408026043, "agent_compute-ego0": 0.01248107990586614, "complete-iteration": 0.16179863458656402, "set_robot_commands": 0.002495014236634036, "deviation-center-line": 0.4270400681641569, "driven_lanedir_consec": 0.7528471039478355, "sim_compute_sim_state": 0.0062751844704869286, "sim_compute_performance-ego0": 0.0021961522389607256}}
set_robot_commands_max0.002495014236634036
set_robot_commands_mean0.0023928501616075058
set_robot_commands_median0.0023904713824527846
set_robot_commands_min0.0022954436448904183
sim_compute_performance-ego0_max0.002305039954522964
sim_compute_performance-ego0_mean0.002182385973152659
sim_compute_performance-ego0_median0.002183235089155154
sim_compute_performance-ego0_min0.0020580337597773625
sim_compute_sim_state_max0.010429234627780072
sim_compute_sim_state_mean0.00896371947326935
sim_compute_sim_state_median0.009575229397405198
sim_compute_sim_state_min0.0062751844704869286
sim_render-ego0_max0.004145530538693951
sim_render-ego0_mean0.004019427964027799
sim_render-ego0_median0.004020477372274546
sim_render-ego0_min0.003891226572868152
simulation-passed1
step_physics_max0.14529307186603546
step_physics_mean0.122006099160869
step_physics_median0.12336575010377726
step_physics_min0.09599982456988598
survival_time_max59.99999999999873
survival_time_mean23.499999999999716
survival_time_min5.549999999999988
No reset possible
5437911558Yishu Malhotra 🇨🇦sim-exercise-2aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-040:44:42
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.0
survival_time_median59.99999999999873
deviation-center-line_median1.2422730096440104
in-drivable-lane_median0.0


other stats
agent_compute-ego0_max0.013537434316693893
agent_compute-ego0_mean0.01310033618560143
agent_compute-ego0_median0.013021778703033676
agent_compute-ego0_min0.012820353019644478
complete-iteration_max0.37093134684725465
complete-iteration_mean0.31131709559970255
complete-iteration_median0.31115542254181927
complete-iteration_min0.2520261904679171
deviation-center-line_max4.053503393024394
deviation-center-line_mean1.731069028434027
deviation-center-line_min0.386226701423694
deviation-heading_max27.859809596736422
deviation-heading_mean14.925250437835436
deviation-heading_median14.309269950178932
deviation-heading_min3.22265225424745
driven_any_max2.6645352591003757e-13
driven_any_mean1.9984014443252818e-13
driven_any_median2.6645352591003757e-13
driven_any_min0.0
driven_lanedir_consec_max0.000286102294921875
driven_lanedir_consec_mean7.152557373046875e-05
driven_lanedir_consec_min0.0
driven_lanedir_max0.000286102294921875
driven_lanedir_mean7.152557373046875e-05
driven_lanedir_median0.0
driven_lanedir_min0.0
get_duckie_state_max1.5029700769175102e-06
get_duckie_state_mean1.4573608608071156e-06
get_duckie_state_median1.4791480706792192e-06
get_duckie_state_min1.3681772249525136e-06
get_robot_state_max0.00400926667784374
get_robot_state_mean0.0039619435180136805
get_robot_state_median0.003989116933125441
get_robot_state_min0.0038602735279600984
get_state_dump_max0.005198270653209321
get_state_dump_mean0.00505249769264812
get_state_dump_median0.005041825384224185
get_state_dump_min0.004928069348934787
get_ui_image_max0.0371759551252354
get_ui_image_mean0.031670171454189024
get_ui_image_median0.03128584556833691
get_ui_image_min0.026933039554846872
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 2.6645352591003757e-13, "get_ui_image": 0.02983536092963048, "step_physics": 0.23002960600523428, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "get_state_dump": 0.005063975284141267, "get_robot_state": 0.003970643860612881, "sim_render-ego0": 0.004017828703919219, "get_duckie_state": 1.4964190252019802e-06, "in-drivable-lane": 0.0, "deviation-heading": 22.66279310353771, "agent_compute-ego0": 0.013045752475303376, "complete-iteration": 0.2996077206013701, "set_robot_commands": 0.002342639418069171, "deviation-center-line": 4.053503393024394, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.009011886995301258, "sim_compute_performance-ego0": 0.0021907963621725546}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.0, "get_ui_image": 0.0371759551252354, "step_physics": 0.29064969595624046, "survival_time": 59.99999999999873, "driven_lanedir": 0.000286102294921875, "get_state_dump": 0.005198270653209321, "get_robot_state": 0.0040075900056380015, "sim_render-ego0": 0.004099434956622858, "get_duckie_state": 1.5029700769175102e-06, "in-drivable-lane": 0.0, "deviation-heading": 27.859809596736422, "agent_compute-ego0": 0.012820353019644478, "complete-iteration": 0.37093134684725465, "set_robot_commands": 0.00234020441199818, "deviation-center-line": 1.0457540566888746, "driven_lanedir_consec": 0.000286102294921875, "sim_compute_sim_state": 0.0123355065059106, "sim_compute_performance-ego0": 0.0022055812918276314}, "LF-norm-techtrack-000-ego0": {"driven_any": 2.6645352591003757e-13, "get_ui_image": 0.03273633020704334, "step_physics": 0.250727205451184, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "get_state_dump": 0.004928069348934787, "get_robot_state": 0.0038602735279600984, "sim_render-ego0": 0.003922936719819767, "get_duckie_state": 1.3681772249525136e-06, "in-drivable-lane": 0.0, "deviation-heading": 3.22265225424745, "agent_compute-ego0": 0.012997804930763976, "complete-iteration": 0.3227031244822684, "set_robot_commands": 0.0023057240431354404, "deviation-center-line": 1.4387919625991463, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.009044004419661878, "sim_compute_performance-ego0": 0.0020841067280003072}, "LF-norm-small_loop-000-ego0": {"driven_any": 2.6645352591003757e-13, "get_ui_image": 0.026933039554846872, "step_physics": 0.18744536819902685, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "get_state_dump": 0.005019675484307104, "get_robot_state": 0.00400926667784374, "sim_render-ego0": 0.004015231311172371, "get_duckie_state": 1.461877116156458e-06, "in-drivable-lane": 0.0, "deviation-heading": 5.955746796820156, "agent_compute-ego0": 0.013537434316693893, "complete-iteration": 0.2520261904679171, "set_robot_commands": 0.0023954468503979026, "deviation-center-line": 0.386226701423694, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.00642119835655854, "sim_compute_performance-ego0": 0.0021544275037652747}}
set_robot_commands_max0.0023954468503979026
set_robot_commands_mean0.0023460036809001736
set_robot_commands_median0.0023414219150336753
set_robot_commands_min0.0023057240431354404
sim_compute_performance-ego0_max0.0022055812918276314
sim_compute_performance-ego0_mean0.002158727971441442
sim_compute_performance-ego0_median0.0021726119329689144
sim_compute_performance-ego0_min0.0020841067280003072
sim_compute_sim_state_max0.0123355065059106
sim_compute_sim_state_mean0.00920314906935807
sim_compute_sim_state_median0.009027945707481567
sim_compute_sim_state_min0.00642119835655854
sim_render-ego0_max0.004099434956622858
sim_render-ego0_mean0.004013857922883554
sim_render-ego0_median0.004016530007545795
sim_render-ego0_min0.003922936719819767
simulation-passed1
step_physics_max0.29064969595624046
step_physics_mean0.23971296890292135
step_physics_median0.24037840572820912
step_physics_min0.18744536819902685
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
5434411571Yishu Malhotra 🇨🇦sim-exercise-2aido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:06:16
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5431011587Yishu Malhotra 🇨🇦exercise_state_estimationaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:05:09
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5430711624Anthony Courchesne 🇨🇦TNetR1aido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:00:40
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 35, in init
              ||     self.check_tensorflow_gpu()
              ||   File "solution.py", line 51, in check_tensorflow_gpu
              ||     raise Exception(msg)
              || Exception: Could not find gpu device.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 35, in init
              || |     self.check_tensorflow_gpu()
              || |   File "solution.py", line 51, in check_tensorflow_gpu
              || |     raise Exception(msg)
              || | Exception: Could not find gpu device.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 35, in init
              ||     self.check_tensorflow_gpu()
              ||   File "solution.py", line 51, in check_tensorflow_gpu
              ||     raise Exception(msg)
              || Exception: Could not find gpu device.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 35, in init
              || |     self.check_tensorflow_gpu()
              || |   File "solution.py", line 51, in check_tensorflow_gpu
              || |     raise Exception(msg)
              || | Exception: Could not find gpu device.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5430411624Anthony Courchesne 🇨🇦TNetR1aido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-040:00:41
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 35, in init
              ||     self.check_tensorflow_gpu()
              ||   File "solution.py", line 51, in check_tensorflow_gpu
              ||     raise Exception(msg)
              || Exception: Could not find gpu device.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 35, in init
              || |     self.check_tensorflow_gpu()
              || |   File "solution.py", line 51, in check_tensorflow_gpu
              || |     raise Exception(msg)
              || | Exception: Could not find gpu device.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 35, in init
              ||     self.check_tensorflow_gpu()
              ||   File "solution.py", line 51, in check_tensorflow_gpu
              ||     raise Exception(msg)
              || Exception: Could not find gpu device.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 35, in init
              || |     self.check_tensorflow_gpu()
              || |   File "solution.py", line 51, in check_tensorflow_gpu
              || |     raise Exception(msg)
              || | Exception: Could not find gpu device.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5422611630Jean-Sébastien Grondin 🇨🇦sim-exercise-2aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-040:25:35
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median4.046519172642444
survival_time_median41.324999999999456
deviation-center-line_median2.507862756087124
in-drivable-lane_median4.099999999999783


other stats
agent_compute-ego0_max0.012940136816579038
agent_compute-ego0_mean0.01267627375805666
agent_compute-ego0_median0.01274151271699776
agent_compute-ego0_min0.012281932781652075
complete-iteration_max0.3068576436681846
complete-iteration_mean0.26453037524175693
complete-iteration_median0.26494310698942825
complete-iteration_min0.22137764331998672
deviation-center-line_max4.2937072839680095
deviation-center-line_mean2.4914406692163564
deviation-center-line_min0.656329880723169
deviation-heading_max30.71590186373302
deviation-heading_mean18.110697884005887
deviation-heading_median18.304045809730177
deviation-heading_min5.1187980528301695
driven_any_max10.86065524127944
driven_any_mean6.89227346735829
driven_any_median7.586590276845787
driven_any_min1.5352580744621451
driven_lanedir_consec_max5.911709295114664
driven_lanedir_consec_mean3.814504044409416
driven_lanedir_consec_min1.2532685372381114
driven_lanedir_max8.808586953845882
driven_lanedir_mean4.979089699960401
driven_lanedir_median4.927251654378804
driven_lanedir_min1.2532685372381114
get_duckie_state_max1.4275337238295889e-06
get_duckie_state_mean1.3991174696671842e-06
get_duckie_state_median1.396706905648247e-06
get_duckie_state_min1.3755223435426532e-06
get_robot_state_max0.003820882848061491
get_robot_state_mean0.003739374643057303
get_robot_state_median0.00371942974803309
get_robot_state_min0.0036977562281015415
get_state_dump_max0.0048474735463290885
get_state_dump_mean0.0047707114494708325
get_state_dump_median0.004751165365959334
get_state_dump_min0.0047330415196355745
get_ui_image_max0.03808116544153273
get_ui_image_mean0.03217098023204943
get_ui_image_median0.03170922864567706
get_ui_image_min0.02718429819531087
in-drivable-lane_max8.950000000000127
in-drivable-lane_mean4.287499999999923
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 10.86065524127944, "get_ui_image": 0.02925902381725454, "step_physics": 0.1798338679647962, "survival_time": 59.99999999999873, "driven_lanedir": 8.808586953845882, "get_state_dump": 0.004740717210539374, "get_robot_state": 0.0036977562281015415, "sim_render-ego0": 0.003779534991833689, "get_duckie_state": 1.3755223435426532e-06, "in-drivable-lane": 1.0499999999999403, "deviation-heading": 30.71590186373302, "agent_compute-ego0": 0.012706079252752831, "complete-iteration": 0.24876637581881636, "set_robot_commands": 0.0022080417080386095, "deviation-center-line": 4.073923972303004, "driven_lanedir_consec": 5.904312806060484, "sim_compute_sim_state": 0.010441248264042762, "sim_compute_performance-ego0": 0.0020101979610624163}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.5352580744621451, "get_ui_image": 0.03808116544153273, "step_physics": 0.22938089149514423, "survival_time": 9.650000000000002, "driven_lanedir": 1.2532685372381114, "get_state_dump": 0.0047616135213792935, "get_robot_state": 0.003700825356945549, "sim_render-ego0": 0.003825024231192992, "get_duckie_state": 1.4169928953819669e-06, "in-drivable-lane": 0.0, "deviation-heading": 5.1187980528301695, "agent_compute-ego0": 0.012281932781652075, "complete-iteration": 0.3068576436681846, "set_robot_commands": 0.002280954233149892, "deviation-center-line": 0.656329880723169, "driven_lanedir_consec": 1.2532685372381114, "sim_compute_sim_state": 0.010414944481603875, "sim_compute_performance-ego0": 0.002038684087930266}, "LF-norm-techtrack-000-ego0": {"driven_any": 4.328318811874022, "get_ui_image": 0.03415943347409958, "step_physics": 0.20609136142394616, "survival_time": 22.650000000000187, "driven_lanedir": 2.232875109113498, "get_state_dump": 0.0047330415196355745, "get_robot_state": 0.003738034139120631, "sim_render-ego0": 0.003938621886501228, "get_duckie_state": 1.376420915914527e-06, "in-drivable-lane": 8.950000000000127, "deviation-heading": 7.50809371686958, "agent_compute-ego0": 0.012776946181242688, "complete-iteration": 0.2811198381600401, "set_robot_commands": 0.002310064395618859, "deviation-center-line": 0.9418015398712434, "driven_lanedir_consec": 2.1887255392244036, "sim_compute_sim_state": 0.011210849631725427, "sim_compute_performance-ego0": 0.0020673432539213073}, "LF-norm-small_loop-000-ego0": {"driven_any": 10.844861741817551, "get_ui_image": 0.02718429819531087, "step_physics": 0.15812968155625062, "survival_time": 59.99999999999873, "driven_lanedir": 7.621628199644109, "get_state_dump": 0.0048474735463290885, "get_robot_state": 0.003820882848061491, "sim_render-ego0": 0.003864023310258724, "get_duckie_state": 1.4275337238295889e-06, "in-drivable-lane": 7.149999999999625, "deviation-heading": 29.099997902590772, "agent_compute-ego0": 0.012940136816579038, "complete-iteration": 0.22137764331998672, "set_robot_commands": 0.0022751312271740713, "deviation-center-line": 4.2937072839680095, "driven_lanedir_consec": 5.911709295114664, "sim_compute_sim_state": 0.006194461493766079, "sim_compute_performance-ego0": 0.0020241755232227334}}
set_robot_commands_max0.002310064395618859
set_robot_commands_mean0.002268547890995358
set_robot_commands_median0.0022780427301619817
set_robot_commands_min0.0022080417080386095
sim_compute_performance-ego0_max0.0020673432539213073
sim_compute_performance-ego0_mean0.002035100206534181
sim_compute_performance-ego0_median0.0020314298055765
sim_compute_performance-ego0_min0.0020101979610624163
sim_compute_sim_state_max0.011210849631725427
sim_compute_sim_state_mean0.009565375967784536
sim_compute_sim_state_median0.01042809637282332
sim_compute_sim_state_min0.006194461493766079
sim_render-ego0_max0.003938621886501228
sim_render-ego0_mean0.0038518011049466583
sim_render-ego0_median0.003844523770725858
sim_render-ego0_min0.003779534991833689
simulation-passed1
step_physics_max0.22938089149514423
step_physics_mean0.1933589506100343
step_physics_median0.1929626146943712
step_physics_min0.15812968155625062
survival_time_max59.99999999999873
survival_time_mean38.07499999999941
survival_time_min9.650000000000002
No reset possible
5418211642Mo Kleit 🇨🇦sim-exercise-2aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-040:18:27
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median2.9157484896013126
survival_time_median23.050000000000193
deviation-center-line_median1.1976128479724877
in-drivable-lane_median7.600000000000094


other stats
agent_compute-ego0_max0.01298535631058064
agent_compute-ego0_mean0.012720399780877877
agent_compute-ego0_median0.012814395606479262
agent_compute-ego0_min0.012267451599972333
complete-iteration_max0.32373621463775637
complete-iteration_mean0.25793663802175354
complete-iteration_median0.2483538003166481
complete-iteration_min0.21130273681596157
deviation-center-line_max1.659441533329164
deviation-center-line_mean1.12977273207345
deviation-center-line_min0.464423699019661
deviation-heading_max7.500996821740706
deviation-heading_mean5.640459933561047
deviation-heading_median6.034037862354257
deviation-heading_min2.992767187794967
driven_any_max15.839419787045363
driven_any_mean7.23758751313428
driven_any_median5.837306610974521
driven_any_min1.4363170435427106
driven_lanedir_consec_max5.014917033726544
driven_lanedir_consec_mean2.951008637078516
driven_lanedir_consec_min0.9576205353848952
driven_lanedir_max5.2199115117205634
driven_lanedir_mean3.3061291072562504
driven_lanedir_median3.5234921909597707
driven_lanedir_min0.9576205353848952
get_duckie_state_max2.563916719876803e-06
get_duckie_state_mean2.348192205018584e-06
get_duckie_state_median2.280435161262764e-06
get_duckie_state_min2.2679817776720063e-06
get_robot_state_max0.0040338329268806246
get_robot_state_mean0.003968630650425112
get_robot_state_median0.004011359334993763
get_robot_state_min0.003817971004832297
get_state_dump_max0.005341771932748648
get_state_dump_mean0.005144342254513151
get_state_dump_median0.005158517549529435
get_state_dump_min0.004918561986245085
get_ui_image_max0.03842811767871563
get_ui_image_mean0.032360060138803215
get_ui_image_median0.031773085076672945
get_ui_image_min0.027465952723151343
in-drivable-lane_max37.799999999998576
in-drivable-lane_mean13.51249999999969
in-drivable-lane_min1.0499999999999965
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 7.609723625774113, "get_ui_image": 0.02978231726574297, "step_physics": 0.16028104750048214, "survival_time": 29.700000000000287, "driven_lanedir": 5.014917033726544, "get_state_dump": 0.00513064400488589, "get_robot_state": 0.004008846523381081, "sim_render-ego0": 0.004051059835097369, "get_duckie_state": 2.2679817776720063e-06, "in-drivable-lane": 8.550000000000122, "deviation-heading": 7.431444517342627, "agent_compute-ego0": 0.01279832655642213, "complete-iteration": 0.23107451831593231, "set_robot_commands": 0.0024157480031502345, "deviation-center-line": 1.659441533329164, "driven_lanedir_consec": 5.014917033726544, "sim_compute_sim_state": 0.01028220473217363, "sim_compute_performance-ego0": 0.002223237622685793}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.4363170435427106, "get_ui_image": 0.03842811767871563, "step_physics": 0.24318980620457575, "survival_time": 6.449999999999985, "driven_lanedir": 0.9576205353848952, "get_state_dump": 0.005341771932748648, "get_robot_state": 0.004013872146606446, "sim_render-ego0": 0.00409551217005803, "get_duckie_state": 2.563916719876803e-06, "in-drivable-lane": 1.0499999999999965, "deviation-heading": 2.992767187794967, "agent_compute-ego0": 0.012830464656536396, "complete-iteration": 0.32373621463775637, "set_robot_commands": 0.0023617285948533276, "deviation-center-line": 0.464423699019661, "driven_lanedir_consec": 0.9576205353848952, "sim_compute_sim_state": 0.011167540917029747, "sim_compute_performance-ego0": 0.0021991601357093226}, "LF-norm-techtrack-000-ego0": {"driven_any": 4.064889596174929, "get_ui_image": 0.033763852887602926, "step_physics": 0.18954138915227176, "survival_time": 16.400000000000098, "driven_lanedir": 2.0320673481929976, "get_state_dump": 0.00518639109417298, "get_robot_state": 0.0040338329268806246, "sim_render-ego0": 0.004061406868931733, "get_duckie_state": 2.291427194891005e-06, "in-drivable-lane": 6.650000000000067, "deviation-heading": 4.636631207365887, "agent_compute-ego0": 0.01298535631058064, "complete-iteration": 0.2656330823173639, "set_robot_commands": 0.002402258499052749, "deviation-center-line": 0.8244193841675594, "driven_lanedir_consec": 2.0320673481929976, "sim_compute_sim_state": 0.011376738910617074, "sim_compute_performance-ego0": 0.002179277704117146}, "LF-norm-small_loop-000-ego0": {"driven_any": 15.839419787045363, "get_ui_image": 0.027465952723151343, "step_physics": 0.14825698318926123, "survival_time": 59.99999999999873, "driven_lanedir": 5.2199115117205634, "get_state_dump": 0.004918561986245085, "get_robot_state": 0.003817971004832297, "sim_render-ego0": 0.003832825216822184, "get_duckie_state": 2.2694431276345235e-06, "in-drivable-lane": 37.799999999998576, "deviation-heading": 7.500996821740706, "agent_compute-ego0": 0.012267451599972333, "complete-iteration": 0.21130273681596157, "set_robot_commands": 0.002279448568771324, "deviation-center-line": 1.5708063117774156, "driven_lanedir_consec": 3.799429631009627, "sim_compute_sim_state": 0.00634899683340106, "sim_compute_performance-ego0": 0.0020187494657518067}}
set_robot_commands_max0.0024157480031502345
set_robot_commands_mean0.002364795916456909
set_robot_commands_median0.002381993546953038
set_robot_commands_min0.002279448568771324
sim_compute_performance-ego0_max0.002223237622685793
sim_compute_performance-ego0_mean0.002155106232066017
sim_compute_performance-ego0_median0.0021892189199132343
sim_compute_performance-ego0_min0.0020187494657518067
sim_compute_sim_state_max0.011376738910617074
sim_compute_sim_state_mean0.009793870348305376
sim_compute_sim_state_median0.010724872824601687
sim_compute_sim_state_min0.00634899683340106
sim_render-ego0_max0.00409551217005803
sim_render-ego0_mean0.004010201022727328
sim_render-ego0_median0.0040562333520145506
sim_render-ego0_min0.003832825216822184
simulation-passed1
step_physics_max0.24318980620457575
step_physics_mean0.18531730651164777
step_physics_median0.17491121832637696
step_physics_min0.14825698318926123
survival_time_max59.99999999999873
survival_time_mean28.137499999999775
survival_time_min6.449999999999985
No reset possible
5415411649Melisande Tengexercise_state_estimationaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-040:09:22
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.114092796255485
survival_time_median12.800000000000049
deviation-center-line_median0.435615037573031
in-drivable-lane_median5.200000000000053


other stats
agent_compute-ego0_max0.013134963985190972
agent_compute-ego0_mean0.012333855613356318
agent_compute-ego0_median0.01224020214505519
agent_compute-ego0_min0.01172005417812393
complete-iteration_max0.24190199608896293
complete-iteration_mean0.1977772734658591
complete-iteration_median0.1934895459793432
complete-iteration_min0.16222800581578714
deviation-center-line_max0.48024724086477966
deviation-center-line_mean0.4150689729349024
deviation-center-line_min0.3087985757287679
deviation-heading_max2.9582566843942626
deviation-heading_mean2.0041605556078466
deviation-heading_median1.7797597329740626
deviation-heading_min1.4988660720889988
driven_any_max5.103190980955817
driven_any_mean2.962567498923788
driven_any_median2.63982526471658
driven_any_min1.4674284853061763
driven_lanedir_consec_max1.940139188230664
driven_lanedir_consec_mean1.2076606657068885
driven_lanedir_consec_min0.6623178820859201
driven_lanedir_max1.940139188230664
driven_lanedir_mean1.238898065515467
driven_lanedir_median1.1171324683294936
driven_lanedir_min0.7811881371722178
get_duckie_state_max1.4503373266028952e-06
get_duckie_state_mean1.387908897135748e-06
get_duckie_state_median1.3847995250761406e-06
get_duckie_state_min1.3316992117878149e-06
get_robot_state_max0.003969396863664899
get_robot_state_mean0.00375525212097796
get_robot_state_median0.003733529287095709
get_robot_state_min0.003584553046055524
get_state_dump_max0.005035392256942131
get_state_dump_mean0.004817667661575022
get_state_dump_median0.00482032088050282
get_state_dump_min0.004594636628352314
get_ui_image_max0.03853612320095885
get_ui_image_mean0.03250256665091143
get_ui_image_median0.03228610386325366
get_ui_image_min0.02690193567617956
in-drivable-lane_max16.90000000000017
in-drivable-lane_mean7.650000000000066
in-drivable-lane_min3.2999999999999883
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 2.6015120905792126, "get_ui_image": 0.028934227649703225, "step_physics": 0.11049694253464616, "survival_time": 13.100000000000051, "driven_lanedir": 1.940139188230664, "get_state_dump": 0.004720196524500394, "get_robot_state": 0.003662745762233951, "sim_render-ego0": 0.003831734675418288, "get_duckie_state": 1.3316992117878149e-06, "in-drivable-lane": 3.600000000000051, "deviation-heading": 1.4988660720889988, "agent_compute-ego0": 0.011915444421224268, "complete-iteration": 0.1777879267137767, "set_robot_commands": 0.0021756230198385143, "deviation-center-line": 0.48024724086477966, "driven_lanedir_consec": 1.940139188230664, "sim_compute_sim_state": 0.009947095080473575, "sim_compute_performance-ego0": 0.0020158091425442424}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.4674284853061763, "get_ui_image": 0.03853612320095885, "step_physics": 0.16346555909300162, "survival_time": 7.599999999999981, "driven_lanedir": 0.7811881371722178, "get_state_dump": 0.004920445236505247, "get_robot_state": 0.0038043128119574655, "sim_render-ego0": 0.003910443362067728, "get_duckie_state": 1.405578812742545e-06, "in-drivable-lane": 3.2999999999999883, "deviation-heading": 1.7281663969409466, "agent_compute-ego0": 0.012564959868886111, "complete-iteration": 0.24190199608896293, "set_robot_commands": 0.00229135993259405, "deviation-center-line": 0.3087985757287679, "driven_lanedir_consec": 0.6623178820859201, "sim_compute_sim_state": 0.010246518390630585, "sim_compute_performance-ego0": 0.002069763108795765}, "LF-norm-techtrack-000-ego0": {"driven_any": 5.103190980955817, "get_ui_image": 0.03563798007680409, "step_physics": 0.1292573444878877, "survival_time": 23.4000000000002, "driven_lanedir": 1.053329735300546, "get_state_dump": 0.005035392256942131, "get_robot_state": 0.003969396863664899, "sim_render-ego0": 0.004172037913601027, "get_duckie_state": 1.4503373266028952e-06, "in-drivable-lane": 16.90000000000017, "deviation-heading": 2.9582566843942626, "agent_compute-ego0": 0.013134963985190972, "complete-iteration": 0.20919116524490977, "set_robot_commands": 0.002413427397640529, "deviation-center-line": 0.4704785520048609, "driven_lanedir_consec": 1.0472503911525288, "sim_compute_sim_state": 0.013253461069135525, "sim_compute_performance-ego0": 0.0022194339776598314}, "LF-norm-small_loop-000-ego0": {"driven_any": 2.678138438853947, "get_ui_image": 0.02690193567617956, "step_physics": 0.10194456149857356, "survival_time": 12.500000000000044, "driven_lanedir": 1.180935201358441, "get_state_dump": 0.004594636628352314, "get_robot_state": 0.003584553046055524, "sim_render-ego0": 0.003637679544578035, "get_duckie_state": 1.364020237409736e-06, "in-drivable-lane": 6.800000000000055, "deviation-heading": 1.8313530690071784, "agent_compute-ego0": 0.01172005417812393, "complete-iteration": 0.16222800581578714, "set_robot_commands": 0.0022539665024594005, "deviation-center-line": 0.40075152314120105, "driven_lanedir_consec": 1.180935201358441, "sim_compute_sim_state": 0.005562471678532452, "sim_compute_performance-ego0": 0.001942171043608768}}
set_robot_commands_max0.002413427397640529
set_robot_commands_mean0.0022835942131331235
set_robot_commands_median0.0022726632175267252
set_robot_commands_min0.0021756230198385143
sim_compute_performance-ego0_max0.0022194339776598314
sim_compute_performance-ego0_mean0.002061794318152152
sim_compute_performance-ego0_median0.0020427861256700035
sim_compute_performance-ego0_min0.001942171043608768
sim_compute_sim_state_max0.013253461069135525
sim_compute_sim_state_mean0.009752386554693036
sim_compute_sim_state_median0.01009680673555208
sim_compute_sim_state_min0.005562471678532452
sim_render-ego0_max0.004172037913601027
sim_render-ego0_mean0.0038879738739162695
sim_render-ego0_median0.003871089018743008
sim_render-ego0_min0.003637679544578035
simulation-passed1
step_physics_max0.16346555909300162
step_physics_mean0.12629110190352727
step_physics_median0.11987714351126692
step_physics_min0.10194456149857356
survival_time_max23.4000000000002
survival_time_mean14.150000000000068
survival_time_min7.599999999999981
No reset possible
5408711671Daniil Lisussim-exercise-2aido5-LF-sim-validationLFv-simerroryesnogpu-prod-040:25:05
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 286, in main
    length_s = await run_episode(
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 577, in run_episode
    await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 304, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 331, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "simulator" aborted:

error in simulator |Exception while handling a message on topic "step".
                   |
                   || Traceback (most recent call last):
                   ||   File "/usr/local/lib/python3.8/site-packages/geometry/manifolds/special_euclidean_group.py", line 73, in belongs
                   ||     self.SOn.belongs(R)
                   ||   File "/usr/local/lib/python3.8/site-packages/geometry/manifolds/special_orthogonal_group.py", line 39, in belongs
                   ||     assert_allclose(det, 1, err_msg="I expect the determinant to be +1.")
                   ||   File "/usr/local/lib/python3.8/site-packages/numpy/testing/_private/utils.py", line 1527, in assert_allclose
                   ||     assert_array_compare(compare, actual, desired, err_msg=str(err_msg),
                   ||   File "/usr/local/lib/python3.8/site-packages/numpy/testing/_private/utils.py", line 764, in assert_array_compare
                   ||     flagged = func_assert_same_pos(x, y, func=isnan, hasval='nan')
                   ||   File "/usr/local/lib/python3.8/site-packages/numpy/testing/_private/utils.py", line 740, in func_assert_same_pos
                   ||     raise AssertionError(msg)
                   || AssertionError:
                   || Not equal to tolerance rtol=1e-07, atol=0
                   || I expect the determinant to be +1.
                   || x and y nan location mismatch:
                   ||  x: array(nan)
                   ||  y: array(1)
                   ||
                   || During handling of the above exception, another exception occurred:
                   ||
                   || Traceback (most recent call last):
                   ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 355, in loop
                   ||     handle_message_node(parsed, receiver0, context0)
                   ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 531, in handle_message_node
                   ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
                   ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
                   ||     f(**kwargs)
                   ||   File "/usr/local/lib/python3.8/site-packages/duckietown_simulator_gym/code.py", line 510, in on_received_step
                   ||     self.update_physics_and_observations(until=data.until, context=context)
                   ||   File "/usr/local/lib/python3.8/site-packages/duckietown_simulator_gym/code.py", line 560, in update_physics_and_observations
                   ||     pc.integrate(delta_time)
                   ||   File "/usr/local/lib/python3.8/site-packages/duckietown_simulator_gym/code.py", line 217, in integrate
                   ||     self.state = self.state.integrate(dt, self.last_commands.wheels)
                   ||   File "/usr/local/lib/python3.8/site-packages/duckietown_world/world_duckietown/dynamics_delay.py", line 74, in integrate
                   ||     state2 = self.state.integrate(dt, use_commands)
                   ||   File "/usr/local/lib/python3.8/site-packages/duckietown_world/world_duckietown/pwm_dynamics.py", line 199, in integrate
                   ||     s1 = GenericKinematicsSE2.integrate(self, dt, commands_se2)
                   ||   File "/usr/local/lib/python3.8/site-packages/duckietown_world/world_duckietown/generic_kinematics.py", line 48, in integrate
                   ||     return GenericKinematicsSE2(c1, t1)
                   ||   File "/usr/local/lib/python3.8/site-packages/duckietown_world/world_duckietown/generic_kinematics.py", line 26, in __init__
                   ||     geo.SE2.belongs(q0)
                   ||   File "/usr/local/lib/python3.8/site-packages/geometry/manifolds/special_euclidean_group.py", line 76, in belongs
                   ||     raise ValueError(msg)
                   || ValueError: The rotation is not a rotation:
                   || [[nan nan]
                   ||  [nan nan]]
                   ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5397811710Yishu Malhotra 🇨🇦sim-exercise-2aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-040:33:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median11.09266200279434
survival_time_median59.99999999999873
deviation-center-line_median3.1850263053607004
in-drivable-lane_median0.1750000000000025


other stats
agent_compute-ego0_max0.013476555194584754
agent_compute-ego0_mean0.012757804444617542
agent_compute-ego0_median0.012860593397212697
agent_compute-ego0_min0.011833475789460016
complete-iteration_max0.2432600608177725
complete-iteration_mean0.20973201364489716
complete-iteration_median0.20275322334851748
complete-iteration_min0.1901615470647812
deviation-center-line_max3.4989223394346425
deviation-center-line_mean2.756274874225975
deviation-center-line_min1.1561245467478565
deviation-heading_max14.690901158371052
deviation-heading_mean11.534752667038214
deviation-heading_median13.27735249283919
deviation-heading_min4.893404524103427
driven_any_max11.947873550764282
driven_any_mean9.7362215587718
driven_any_median11.56002469449624
driven_any_min3.876963295330442
driven_lanedir_consec_max11.442510650393787
driven_lanedir_consec_mean8.995096674928543
driven_lanedir_consec_min2.352552043731708
driven_lanedir_max11.442510650393787
driven_lanedir_mean8.995096674928543
driven_lanedir_median11.09266200279434
driven_lanedir_min2.352552043731708
get_duckie_state_max1.5124678611755371e-06
get_duckie_state_mean1.455516826600258e-06
get_duckie_state_median1.45344015561373e-06
get_duckie_state_min1.4027191339980354e-06
get_robot_state_max0.004282702017981841
get_robot_state_mean0.004128429782790109
get_robot_state_median0.004188923852256096
get_robot_state_min0.003853169408666403
get_state_dump_max0.005804826377531968
get_state_dump_mean0.005330690995888548
get_state_dump_median0.005250056347283198
get_state_dump_min0.005017824911455826
get_ui_image_max0.03722785791687723
get_ui_image_mean0.031605196935379304
get_ui_image_median0.03065355989359301
get_ui_image_min0.02788581003745397
in-drivable-lane_max8.800000000000125
in-drivable-lane_mean2.2875000000000325
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 11.947873550764282, "get_ui_image": 0.02894557524878814, "step_physics": 0.12700290584643617, "survival_time": 59.99999999999873, "driven_lanedir": 11.442510650393787, "get_state_dump": 0.005017824911455826, "get_robot_state": 0.003853169408666403, "sim_render-ego0": 0.00397709922727002, "get_duckie_state": 1.4241589396124974e-06, "in-drivable-lane": 0.0, "deviation-heading": 14.690901158371052, "agent_compute-ego0": 0.011833475789460016, "complete-iteration": 0.19492577990326257, "set_robot_commands": 0.002239422635373029, "deviation-center-line": 3.1819775198712574, "driven_lanedir_consec": 11.442510650393787, "sim_compute_sim_state": 0.009853885731629588, "sim_compute_performance-ego0": 0.0021079956343727844}, "LF-norm-zigzag-000-ego0": {"driven_any": 11.53534729438996, "get_ui_image": 0.03722785791687723, "step_physics": 0.15968955446540267, "survival_time": 59.99999999999873, "driven_lanedir": 11.028692807919256, "get_state_dump": 0.005210254114930775, "get_robot_state": 0.004131635162455156, "sim_render-ego0": 0.004229480876811438, "get_duckie_state": 1.4827213716149628e-06, "in-drivable-lane": 0.350000000000005, "deviation-heading": 13.355920791600187, "agent_compute-ego0": 0.012788656252210682, "complete-iteration": 0.2432600608177725, "set_robot_commands": 0.002448409125767183, "deviation-center-line": 3.4989223394346425, "driven_lanedir_consec": 11.028692807919256, "sim_compute_sim_state": 0.0150371933856872, "sim_compute_performance-ego0": 0.0023972781671274712}, "LF-norm-techtrack-000-ego0": {"driven_any": 11.58470209460252, "get_ui_image": 0.032361544538397874, "step_physics": 0.13208066633003737, "survival_time": 59.99999999999873, "driven_lanedir": 11.156631197669425, "get_state_dump": 0.005804826377531968, "get_robot_state": 0.004282702017981841, "sim_render-ego0": 0.004058821016703915, "get_duckie_state": 1.4027191339980354e-06, "in-drivable-lane": 0.0, "deviation-heading": 13.198784194078192, "agent_compute-ego0": 0.013476555194584754, "complete-iteration": 0.2105806667937724, "set_robot_commands": 0.002653471734700453, "deviation-center-line": 3.1880750908501434, "driven_lanedir_consec": 11.156631197669425, "sim_compute_sim_state": 0.013500043097185553, "sim_compute_performance-ego0": 0.0022657036682052676}, "LF-norm-small_loop-000-ego0": {"driven_any": 3.876963295330442, "get_ui_image": 0.02788581003745397, "step_physics": 0.12379868080218633, "survival_time": 23.950000000000205, "driven_lanedir": 2.352552043731708, "get_state_dump": 0.0052898585796356205, "get_robot_state": 0.004246212542057037, "sim_render-ego0": 0.004216093818346659, "get_duckie_state": 1.5124678611755371e-06, "in-drivable-lane": 8.800000000000125, "deviation-heading": 4.893404524103427, "agent_compute-ego0": 0.012932530542214712, "complete-iteration": 0.1901615470647812, "set_robot_commands": 0.0025528709093729654, "deviation-center-line": 1.1561245467478565, "driven_lanedir_consec": 2.352552043731708, "sim_compute_sim_state": 0.006806485851605733, "sim_compute_performance-ego0": 0.00233327845732371}}
set_robot_commands_max0.002653471734700453
set_robot_commands_mean0.002473543601303408
set_robot_commands_median0.002500640017570074
set_robot_commands_min0.002239422635373029
sim_compute_performance-ego0_max0.0023972781671274712
sim_compute_performance-ego0_mean0.0022760639817573083
sim_compute_performance-ego0_median0.0022994910627644888
sim_compute_performance-ego0_min0.0021079956343727844
sim_compute_sim_state_max0.0150371933856872
sim_compute_sim_state_mean0.011299402016527018
sim_compute_sim_state_median0.011676964414407569
sim_compute_sim_state_min0.006806485851605733
sim_render-ego0_max0.004229480876811438
sim_render-ego0_mean0.004120373734783008
sim_render-ego0_median0.0041374574175252875
sim_render-ego0_min0.00397709922727002
simulation-passed1
step_physics_max0.15968955446540267
step_physics_mean0.13564295186101563
step_physics_median0.12954178608823677
step_physics_min0.12379868080218633
survival_time_max59.99999999999873
survival_time_mean50.9874999999991
survival_time_min23.950000000000205
No reset possible
5390811729Yishu Malhotra 🇨🇦sim-exercise-2aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-040:26:17
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.869064475659838
survival_time_median49.77499999999931
deviation-center-line_median0.914007491848232
in-drivable-lane_median26.37499999999945


other stats
agent_compute-ego0_max0.013176634150877484
agent_compute-ego0_mean0.012746709374061615
agent_compute-ego0_median0.012678755625446824
agent_compute-ego0_min0.012452692094475332
complete-iteration_max0.2834378955810051
complete-iteration_mean0.22389618576170983
complete-iteration_median0.2099152074864663
complete-iteration_min0.1923164324929016
deviation-center-line_max2.3311101415752726
deviation-center-line_mean1.101240158517741
deviation-center-line_min0.24583550879922733
deviation-heading_max8.478659451594144
deviation-heading_mean4.936405601220808
deviation-heading_median5.177647373470299
deviation-heading_min0.9116682063484914
driven_any_max10.705656314114764
driven_any_mean7.0273229015088665
driven_any_median8.397747940884118
driven_any_min0.6081394101524684
driven_lanedir_consec_max4.098245300329343
driven_lanedir_consec_mean2.0491899572097827
driven_lanedir_consec_min0.36038557719011255
driven_lanedir_max4.980166353734681
driven_lanedir_mean2.5830801760747732
driven_lanedir_median2.495884386687149
driven_lanedir_min0.36038557719011255
get_duckie_state_max1.58317083125309e-06
get_duckie_state_mean1.4338703943621616e-06
get_duckie_state_median1.395217461572861e-06
get_duckie_state_min1.3618758230498342e-06
get_robot_state_max0.004289998102942474
get_robot_state_mean0.004062208853497657
get_robot_state_median0.003996828608199502
get_robot_state_min0.003965180094649152
get_state_dump_max0.0053470567501554085
get_state_dump_mean0.005191946120158086
get_state_dump_median0.005168106706795288
get_state_dump_min0.005084514316886363
get_ui_image_max0.037640823581354406
get_ui_image_mean0.03289914561020159
get_ui_image_median0.03273372864544541
get_ui_image_min0.02848830156856113
in-drivable-lane_max53.19999999999884
in-drivable-lane_mean27.34999999999943
in-drivable-lane_min3.4499999999999877
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 10.705656314114764, "get_ui_image": 0.029836376541163105, "step_physics": 0.12373394314990652, "survival_time": 59.99999999999873, "driven_lanedir": 4.025075920749424, "get_state_dump": 0.00518555605441307, "get_robot_state": 0.004026932283603182, "sim_render-ego0": 0.004105998316374945, "get_duckie_state": 1.58317083125309e-06, "in-drivable-lane": 39.89999999999907, "deviation-heading": 5.476723420684913, "agent_compute-ego0": 0.012582716199380967, "complete-iteration": 0.19312001167983436, "set_robot_commands": 0.0024159675236050037, "deviation-center-line": 1.31480178236387, "driven_lanedir_consec": 2.8857201492533227, "sim_compute_sim_state": 0.008915302854692013, "sim_compute_performance-ego0": 0.0022202282523632448}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.6081394101524684, "get_ui_image": 0.037640823581354406, "step_physics": 0.20379653015756996, "survival_time": 6.099999999999986, "driven_lanedir": 0.36038557719011255, "get_state_dump": 0.005150657359177504, "get_robot_state": 0.003965180094649152, "sim_render-ego0": 0.004022780472670144, "get_duckie_state": 1.3859291386798146e-06, "in-drivable-lane": 3.4499999999999877, "deviation-heading": 0.9116682063484914, "agent_compute-ego0": 0.01277479505151268, "complete-iteration": 0.2834378955810051, "set_robot_commands": 0.0023935771569973084, "deviation-center-line": 0.24583550879922733, "driven_lanedir_consec": 0.36038557719011255, "sim_compute_sim_state": 0.011454630673416262, "sim_compute_performance-ego0": 0.0021458776985726707}, "LF-norm-techtrack-000-ego0": {"driven_any": 9.93112313379931, "get_ui_image": 0.035631080749727706, "step_physics": 0.1469905013148731, "survival_time": 59.99999999999873, "driven_lanedir": 0.9666928526248748, "get_state_dump": 0.0053470567501554085, "get_robot_state": 0.004289998102942474, "sim_render-ego0": 0.004311088320615389, "get_duckie_state": 1.4045057844659074e-06, "in-drivable-lane": 53.19999999999884, "deviation-heading": 4.878571326255685, "agent_compute-ego0": 0.013176634150877484, "complete-iteration": 0.22671040329309824, "set_robot_commands": 0.002556927297434938, "deviation-center-line": 0.5132132013325938, "driven_lanedir_consec": 0.8524088020663536, "sim_compute_sim_state": 0.011886709238666976, "sim_compute_performance-ego0": 0.0024219645151587746}, "LF-norm-small_loop-000-ego0": {"driven_any": 6.864372747968925, "get_ui_image": 0.02848830156856113, "step_physics": 0.12691100558849297, "survival_time": 39.54999999999989, "driven_lanedir": 4.980166353734681, "get_state_dump": 0.005084514316886363, "get_robot_state": 0.003966724932795823, "sim_render-ego0": 0.004070744972036342, "get_duckie_state": 1.3618758230498342e-06, "in-drivable-lane": 12.849999999999824, "deviation-heading": 8.478659451594144, "agent_compute-ego0": 0.012452692094475332, "complete-iteration": 0.1923164324929016, "set_robot_commands": 0.002369453509648641, "deviation-center-line": 2.3311101415752726, "driven_lanedir_consec": 4.098245300329343, "sim_compute_sim_state": 0.006756340313439417, "sim_compute_performance-ego0": 0.002124960675384059}}
set_robot_commands_max0.002556927297434938
set_robot_commands_mean0.0024339813719214728
set_robot_commands_median0.002404772340301156
set_robot_commands_min0.002369453509648641
sim_compute_performance-ego0_max0.0024219645151587746
sim_compute_performance-ego0_mean0.0022282577853696877
sim_compute_performance-ego0_median0.0021830529754679577
sim_compute_performance-ego0_min0.002124960675384059
sim_compute_sim_state_max0.011886709238666976
sim_compute_sim_state_mean0.009753245770053667
sim_compute_sim_state_median0.010184966764054138
sim_compute_sim_state_min0.006756340313439417
sim_render-ego0_max0.004311088320615389
sim_render-ego0_mean0.004127653020424205
sim_render-ego0_median0.004088371644205644
sim_render-ego0_min0.004022780472670144
simulation-passed1
step_physics_max0.20379653015756996
step_physics_mean0.15035799505271064
step_physics_median0.13695075345168303
step_physics_min0.12373394314990652
survival_time_max59.99999999999873
survival_time_mean41.41249999999933
survival_time_min6.099999999999986
No reset possible