Duckietown Challenges Home Challenges Submissions

Evaluator 2119

ID2119
evaluatorreg05-9e3e48edb8ad-1
ownerI don't have one 😀
machinearchimede
processreg05-9e3e48edb8ad-1
versiond-c:6.1.19;d-c-r:6.0.48
first heard
last heard
statusinactive
# evaluating
# success3 36087
# timeout1 36103
# failed
# error2 36084
# aborted1 36094
# host-error
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
361039975Bea Baselines 🐤template-pytorchaido5-LFVI_multi-sim-validationLFVIv-simtimeoutnoreg05-9e3e48edb8ad-11:05:57
I can see how the jo [...]
I can see how the job 36103 is timeout because passed 3957 seconds and the timeout is 3600.0.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
360989938Bea Baselines 🐤template-tensorflowaido5-LFVI-sim-testingLFVIt-simerrornoreg05-9e3e48edb8ad-10:01:17
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "experiment_manager.py", line 196, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 141, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 305, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego" aborted with the following error:

error in ego |Unexpected error:
             |
             || Traceback (most recent call last):
             ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
             ||     call_if_fun_exists(node, "init", context=context_data)
             ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
             ||     f(**kwargs)
             ||   File "solution.py", line 32, in init
             ||     raise Exception(msg)
             || Exception: Could not find gpu device.
             ||
             || The above exception was the direct cause of the following exception:
             ||
             || Traceback (most recent call last):
             ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
             ||     raise Exception(msg) from e
             || Exception: Exception while calling the node's init() function.
             ||
             || | Traceback (most recent call last):
             || |   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
             || |     call_if_fun_exists(node, "init", context=context_data)
             || |   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
             || |     f(**kwargs)
             || |   File "solution.py", line 32, in init
             || |     raise Exception(msg)
             || | Exception: Could not find gpu device.
             || |
             ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 678, in scoring_context
    yield cie
  File "experiment_manager.py", line 655, in go
    wrap(cie)
  File "experiment_manager.py", line 640, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 311, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
360949929Bea Baselines 🐤template-pytorchaido5-LFVI-sim-testingLFVIt-simabortednoreg05-9e3e48edb8ad-10:03:53
KeyboardInterrupt: T [...]
KeyboardInterrupt:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.48-py3.8.egg/duckietown_challenges_runner/runner.py", line 1065, in run_one
    heartbeat()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.48-py3.8.egg/duckietown_challenges_runner/runner.py", line 525, in heartbeat
    raise KeyboardInterrupt(msg)
KeyboardInterrupt: The server told us to abort the job because: The challenge has been updated.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
360929931Bea Baselines 🐤template-tensorflowaido5-LFP-sim-validationLFP-simsuccessnoreg05-9e3e48edb8ad-10:02:44
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median1.3750000000000004
in-drivable-lane_median0.2500000000000002
driven_lanedir_consec_median0.37444621198825995
deviation-center-line_median0.07425038070058781


other stats
agent_compute-ego_max0.031009150402886525
agent_compute-ego_mean0.02891331718936026
agent_compute-ego_median0.029010040026444653
agent_compute-ego_min0.02662403830166521
complete-iteration_max0.2814442685672215
complete-iteration_mean0.2755736064465943
complete-iteration_median0.278611437357389
complete-iteration_min0.26362728250437767
deviation-center-line_max0.13842777162754183
deviation-center-line_mean0.08718325930633847
deviation-center-line_min0.0618045041966364
deviation-heading_max0.3413218493577321
deviation-heading_mean0.3123917106616967
deviation-heading_median0.3234874698163639
deviation-heading_min0.26127005365632694
driven_any_max1.2684656120492492
driven_any_mean0.6792968430235158
driven_any_median0.5107180271847197
driven_any_min0.4272857056753746
driven_lanedir_consec_max0.6424270288897922
driven_lanedir_consec_mean0.4232988778265993
driven_lanedir_consec_min0.30187605844008525
driven_lanedir_max0.6424270288897922
driven_lanedir_mean0.4232988778265993
driven_lanedir_median0.37444621198825995
driven_lanedir_min0.30187605844008525
get_state_dump_max0.0181579956641564
get_state_dump_mean0.01716595322209897
get_state_dump_median0.01695646822452545
get_state_dump_min0.016592880775188577
get_ui_image_max0.05114494521042396
get_ui_image_mean0.050444516964078895
get_ui_image_median0.050348574519157405
get_ui_image_min0.04993597360757681
in-drivable-lane_max1.1999999999999973
in-drivable-lane_mean0.46249999999999947
in-drivable-lane_min0.15000000000000013
per-episodes
details{"ETHZ_autolab_technical_track-sc0-0-ego": {"driven_any": 0.4615870158298592, "get_ui_image": 0.04993597360757681, "step_physics": 0.11848811919872576, "survival_time": 1.3000000000000005, "driven_lanedir": 0.30187605844008525, "get_state_dump": 0.0181579956641564, "sim_render-ego": 0.009037036162156325, "in-drivable-lane": 0.30000000000000027, "agent_compute-ego": 0.02976075502542349, "deviation-heading": 0.26127005365632694, "complete-iteration": 0.28063361461345965, "set_robot_commands": 0.007494541314932017, "deviation-center-line": 0.0618045041966364, "driven_lanedir_consec": 0.30187605844008525, "sim_compute_sim_state": 0.01892434633695162, "sim_compute_performance-ego": 0.006219726342421312, "sim_compute_robot_state-ego": 0.011247772436875563, "sim_compute_robot_state-parked0": 0.011159805151132436}, "ETHZ_autolab_technical_track-sc1-0-ego": {"driven_any": 0.5598490385395802, "get_ui_image": 0.05114494521042396, "step_physics": 0.11369340173129378, "survival_time": 1.4500000000000006, "driven_lanedir": 0.43261011094099466, "get_state_dump": 0.016592880775188577, "sim_render-ego": 0.008325954963421, "in-drivable-lane": 0.20000000000000015, "agent_compute-ego": 0.02662403830166521, "deviation-heading": 0.34091206982542915, "complete-iteration": 0.26362728250437767, "set_robot_commands": 0.006343734675440295, "deviation-center-line": 0.08334250564792163, "driven_lanedir_consec": 0.43261011094099466, "sim_compute_sim_state": 0.013554589501742658, "sim_compute_performance-ego": 0.005995142048802869, "sim_compute_robot_state-ego": 0.01070510930028455, "sim_compute_robot_state-parked0": 0.01045402165116935}, "ETHZ_autolab_technical_track-sc2-0-ego": {"driven_any": 0.4272857056753746, "get_ui_image": 0.050561685562133786, "step_physics": 0.11571019172668456, "survival_time": 1.2500000000000004, "driven_lanedir": 0.31628231303552523, "get_state_dump": 0.01687575340270996, "sim_render-ego": 0.008768348693847657, "in-drivable-lane": 0.15000000000000013, "agent_compute-ego": 0.02825932502746582, "deviation-heading": 0.3413218493577321, "complete-iteration": 0.27658926010131835, "set_robot_commands": 0.007416915893554687, "deviation-center-line": 0.065158255753254, "driven_lanedir_consec": 0.31628231303552523, "sim_compute_sim_state": 0.020129518508911134, "sim_compute_performance-ego": 0.005887956619262696, "sim_compute_robot_state-ego": 0.01143132209777832, "sim_compute_robot_state-parked0": 0.011340084075927737}, "ETHZ_autolab_technical_track-sc3-0-ego": {"driven_any": 1.2684656120492492, "get_ui_image": 0.05013546347618103, "step_physics": 0.11662574325289043, "survival_time": 2.799999999999998, "driven_lanedir": 0.6424270288897922, "get_state_dump": 0.017037183046340942, "sim_render-ego": 0.008851770843778337, "in-drivable-lane": 1.1999999999999973, "agent_compute-ego": 0.031009150402886525, "deviation-heading": 0.3060628698072986, "complete-iteration": 0.2814442685672215, "set_robot_commands": 0.0070839183671133855, "deviation-center-line": 0.13842777162754183, "driven_lanedir_consec": 0.6424270288897922, "sim_compute_sim_state": 0.02277395554951259, "sim_compute_performance-ego": 0.006209003073828561, "sim_compute_robot_state-ego": 0.010724574327468872, "sim_compute_robot_state-parked0": 0.010793289967945645}}
set_robot_commands_max0.007494541314932017
set_robot_commands_mean0.007084777562760097
set_robot_commands_median0.007250417130334036
set_robot_commands_min0.006343734675440295
sim_compute_performance-ego_max0.006219726342421312
sim_compute_performance-ego_mean0.006077957021078859
sim_compute_performance-ego_median0.0061020725613157144
sim_compute_performance-ego_min0.005887956619262696
sim_compute_robot_state-ego_max0.01143132209777832
sim_compute_robot_state-ego_mean0.011027194540601826
sim_compute_robot_state-ego_median0.010986173382172218
sim_compute_robot_state-ego_min0.01070510930028455
sim_compute_robot_state-parked0_max0.011340084075927737
sim_compute_robot_state-parked0_mean0.010936800211543791
sim_compute_robot_state-parked0_median0.01097654755953904
sim_compute_robot_state-parked0_min0.01045402165116935
sim_compute_sim_state_max0.02277395554951259
sim_compute_sim_state_mean0.0188456024742795
sim_compute_sim_state_median0.019526932422931374
sim_compute_sim_state_min0.013554589501742658
sim_render-ego_max0.009037036162156325
sim_render-ego_mean0.00874577766580083
sim_render-ego_median0.008810059768812998
sim_render-ego_min0.008325954963421
simulation-passed1
step_physics_max0.11848811919872576
step_physics_mean0.11612936397739863
step_physics_median0.1161679674897875
step_physics_min0.11369340173129378
survival_time_max2.799999999999998
survival_time_mean1.7
survival_time_min1.2500000000000004
No reset possible
360909932Bea Baselines 🐤template-tensorflowaido5-LF-sim-validationLFv-simsuccessnoreg05-9e3e48edb8ad-10:04:23
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.32185398136116283
survival_time_median1.4250000000000007
deviation-center-line_median0.07354361942868184
in-drivable-lane_median0.4000000000000003


other stats
agent_compute-ego_max0.031270112991333004
agent_compute-ego_mean0.0291781493773063
agent_compute-ego_median0.029104333917299902
agent_compute-ego_min0.02723381668329239
complete-iteration_max0.2725636164347331
complete-iteration_mean0.26959509519239266
complete-iteration_median0.2710390027364095
complete-iteration_min0.2637387588620186
deviation-center-line_max0.3230091739203326
deviation-center-line_mean0.1331355889954076
deviation-center-line_min0.06244594320393397
deviation-heading_max2.226496717186771
deviation-heading_mean0.7473697876487595
deviation-heading_median0.2838152224385325
deviation-heading_min0.19535198853120245
driven_any_max6.70564355753036
driven_any_mean2.046516885812114
driven_any_median0.5335487673671593
driven_any_min0.4133264509837774
driven_lanedir_consec_max1.185811419100096
driven_lanedir_consec_mean0.5266772180063328
driven_lanedir_consec_min0.2771894902029097
driven_lanedir_max1.185811419100096
driven_lanedir_mean0.5266772180063328
driven_lanedir_median0.32185398136116283
driven_lanedir_min0.2771894902029097
get_state_dump_max0.019001798629760744
get_state_dump_mean0.017978628675142924
get_state_dump_median0.018208669622739156
get_state_dump_min0.01649537682533264
get_ui_image_max0.05018056184053421
get_ui_image_mean0.048199162061015766
get_ui_image_median0.04784088492393494
get_ui_image_min0.046934316555658974
in-drivable-lane_max11.150000000000052
in-drivable-lane_mean3.050000000000013
in-drivable-lane_min0.2500000000000001
per-episodes
details{"ETHZ_autolab_technical_track-sc0-0-ego": {"driven_any": 0.4371635894643048, "get_ui_image": 0.04792426109313965, "step_physics": 0.11037742614746092, "survival_time": 1.2500000000000004, "driven_lanedir": 0.2990587466868482, "get_state_dump": 0.019001798629760744, "sim_render-ego": 0.009437780380249025, "in-drivable-lane": 0.2500000000000002, "agent_compute-ego": 0.031270112991333004, "deviation-heading": 0.26668300969771375, "complete-iteration": 0.27209200859069826, "set_robot_commands": 0.0070844745635986325, "deviation-center-line": 0.06244594320393397, "driven_lanedir_consec": 0.2990587466868482, "sim_compute_sim_state": 0.018982114791870116, "sim_compute_performance-ego": 0.0063245010375976565, "sim_compute_robot_state-ego": 0.010725450515747073, "sim_compute_robot_state-parked0": 0.010744781494140624}, "ETHZ_autolab_technical_track-sc1-0-ego": {"driven_any": 0.6299339452700138, "get_ui_image": 0.05018056184053421, "step_physics": 0.11281781643629074, "survival_time": 1.6000000000000008, "driven_lanedir": 0.3446492160354775, "get_state_dump": 0.01649537682533264, "sim_render-ego": 0.007817573845386505, "in-drivable-lane": 0.5500000000000005, "agent_compute-ego": 0.02723381668329239, "deviation-heading": 0.3009474351793513, "complete-iteration": 0.2637387588620186, "set_robot_commands": 0.006667628884315491, "deviation-center-line": 0.07388497485796884, "driven_lanedir_consec": 0.3446492160354775, "sim_compute_sim_state": 0.016684219241142273, "sim_compute_performance-ego": 0.005879424512386322, "sim_compute_robot_state-ego": 0.00991962105035782, "sim_compute_robot_state-parked0": 0.009833991527557371}, "ETHZ_autolab_technical_track-sc2-0-ego": {"driven_any": 6.70564355753036, "get_ui_image": 0.04775750875473023, "step_physics": 0.11120754400889078, "survival_time": 14.950000000000076, "driven_lanedir": 1.185811419100096, "get_state_dump": 0.018001838525136312, "sim_render-ego": 0.008899238109588623, "in-drivable-lane": 11.150000000000052, "agent_compute-ego": 0.030241026083628338, "deviation-heading": 2.226496717186771, "complete-iteration": 0.26998599688212077, "set_robot_commands": 0.007397080262502034, "deviation-center-line": 0.3230091739203326, "driven_lanedir_consec": 1.185811419100096, "sim_compute_sim_state": 0.01798829952875773, "sim_compute_performance-ego": 0.006085151831309, "sim_compute_robot_state-ego": 0.011230950355529784, "sim_compute_robot_state-parked0": 0.0109631085395813}, "ETHZ_autolab_technical_track-sc3-0-ego": {"driven_any": 0.4133264509837774, "get_ui_image": 0.046934316555658974, "step_physics": 0.11370534698168436, "survival_time": 1.2000000000000004, "driven_lanedir": 0.2771894902029097, "get_state_dump": 0.018415500720342, "sim_render-ego": 0.008767743905385336, "in-drivable-lane": 0.2500000000000001, "agent_compute-ego": 0.027967641750971477, "deviation-heading": 0.19535198853120245, "complete-iteration": 0.2725636164347331, "set_robot_commands": 0.007086038589477539, "deviation-center-line": 0.07320226399939485, "driven_lanedir_consec": 0.2771894902029097, "sim_compute_sim_state": 0.02038717269897461, "sim_compute_performance-ego": 0.006193588177363078, "sim_compute_robot_state-ego": 0.011403530836105348, "sim_compute_robot_state-parked0": 0.011491854985555014}}
set_robot_commands_max0.007397080262502034
set_robot_commands_mean0.007058805574973424
set_robot_commands_median0.007085256576538086
set_robot_commands_min0.006667628884315491
sim_compute_performance-ego_max0.0063245010375976565
sim_compute_performance-ego_mean0.006120666389664015
sim_compute_performance-ego_median0.006139370004336039
sim_compute_performance-ego_min0.005879424512386322
sim_compute_robot_state-ego_max0.011403530836105348
sim_compute_robot_state-ego_mean0.010819888189435006
sim_compute_robot_state-ego_median0.010978200435638429
sim_compute_robot_state-ego_min0.00991962105035782
sim_compute_robot_state-parked0_max0.011491854985555014
sim_compute_robot_state-parked0_mean0.010758434136708576
sim_compute_robot_state-parked0_median0.01085394501686096
sim_compute_robot_state-parked0_min0.009833991527557371
sim_compute_sim_state_max0.02038717269897461
sim_compute_sim_state_mean0.018510451565186183
sim_compute_sim_state_median0.018485207160313925
sim_compute_sim_state_min0.016684219241142273
sim_render-ego_max0.009437780380249025
sim_render-ego_mean0.008730584060152372
sim_render-ego_median0.008833491007486979
sim_render-ego_min0.007817573845386505
simulation-passed1
step_physics_max0.11370534698168436
step_physics_mean0.11202703339358172
step_physics_median0.11201268022259075
step_physics_min0.11037742614746092
survival_time_max14.950000000000076
survival_time_mean4.7500000000000195
survival_time_min1.2000000000000004
No reset possible
360879934Bea Baselines 🐤template-tensorflowaido5-LFV-sim-validationLFVv-simsuccessnoreg05-9e3e48edb8ad-10:04:09
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.4299710640914556
survival_time_median1.6500000000000008
deviation-center-line_median0.0947314890507628
in-drivable-lane_median0.25000000000000017


other stats
agent_compute-ego_max0.04660642147064209
agent_compute-ego_mean0.0440859816212585
agent_compute-ego_median0.04375167687733968
agent_compute-ego_min0.04223415125971255
complete-iteration_max0.4533426849930375
complete-iteration_mean0.4429112383918312
complete-iteration_median0.44834147860856566
complete-iteration_min0.4216193113571558
deviation-center-line_max0.14656650223191586
deviation-center-line_mean0.10081938427184647
deviation-center-line_min0.06724805675394444
deviation-heading_max0.8355947377463032
deviation-heading_mean0.4600743394009068
deviation-heading_median0.40429662305994585
deviation-heading_min0.1961093737374323
driven_any_max2.491426417931044
driven_any_mean1.0361406187382367
driven_any_median0.6362053269217316
driven_any_min0.3807254031784394
driven_lanedir_consec_max0.6423572123179142
driven_lanedir_consec_mean0.442195395749215
driven_lanedir_consec_min0.2664822424960347
driven_lanedir_max0.6423572123179142
driven_lanedir_mean0.442195395749215
driven_lanedir_median0.4299710640914556
driven_lanedir_min0.2664822424960347
get_state_dump_max0.03895464068964908
get_state_dump_mean0.03679997261440175
get_state_dump_median0.03652637565680837
get_state_dump_min0.035192498454341185
get_ui_image_max0.07885761428297612
get_ui_image_mean0.07364286578908451
get_ui_image_median0.07376513046416802
get_ui_image_min0.0681835879450259
in-drivable-lane_max4.04999999999999
in-drivable-lane_mean1.1749999999999976
in-drivable-lane_min0.15000000000000013
per-episodes
details{"ETHZ_autolab_technical_track-sc0-0-ego": {"driven_any": 0.4894276499978598, "get_ui_image": 0.07682851508811668, "step_physics": 0.1892991684101246, "survival_time": 1.3500000000000003, "driven_lanedir": 0.3285026434472673, "get_state_dump": 0.035192498454341185, "sim_render-ego": 0.009046104219224716, "in-drivable-lane": 0.30000000000000027, "agent_compute-ego": 0.04346736272176107, "deviation-heading": 0.26421685569527026, "complete-iteration": 0.4533426849930375, "set_robot_commands": 0.00653866485313133, "deviation-center-line": 0.06724805675394444, "driven_lanedir_consec": 0.3285026434472673, "sim_compute_sim_state": 0.01927104702702275, "sim_compute_performance-ego": 0.006391799008404767, "sim_compute_robot_state-ego": 0.011139145603886355, "sim_compute_robot_state-npc0": 0.011191518218429, "sim_compute_robot_state-npc1": 0.011644884392067236, "sim_compute_robot_state-npc2": 0.010682618176495589, "sim_compute_robot_state-npc3": 0.011078437169392904, "sim_compute_robot_state-parked0": 0.011229426772506148}, "ETHZ_autolab_technical_track-sc1-0-ego": {"driven_any": 0.3807254031784394, "get_ui_image": 0.0681835879450259, "step_physics": 0.19733271391495413, "survival_time": 1.1500000000000004, "driven_lanedir": 0.2664822424960347, "get_state_dump": 0.03527156166408373, "sim_render-ego": 0.009043849032858145, "in-drivable-lane": 0.20000000000000007, "agent_compute-ego": 0.04223415125971255, "deviation-heading": 0.1961093737374323, "complete-iteration": 0.44334048810212506, "set_robot_commands": 0.006651474081951639, "deviation-center-line": 0.06983282949182971, "driven_lanedir_consec": 0.2664822424960347, "sim_compute_sim_state": 0.017230303391166355, "sim_compute_performance-ego": 0.005988276523092519, "sim_compute_robot_state-ego": 0.009545968926471212, "sim_compute_robot_state-npc0": 0.010384487069171408, "sim_compute_robot_state-npc1": 0.010347957196442978, "sim_compute_robot_state-npc2": 0.009910500567892324, "sim_compute_robot_state-npc3": 0.010415025379346764, "sim_compute_robot_state-parked0": 0.010422820630280868}, "ETHZ_autolab_technical_track-sc2-0-ego": {"driven_any": 0.7829830038456033, "get_ui_image": 0.07070174584021935, "step_physics": 0.1588425147227752, "survival_time": 1.950000000000001, "driven_lanedir": 0.6423572123179142, "get_state_dump": 0.03778118964953301, "sim_render-ego": 0.008948252751277043, "in-drivable-lane": 0.15000000000000013, "agent_compute-ego": 0.0440359910329183, "deviation-heading": 0.5443763904246215, "complete-iteration": 0.4216193113571558, "set_robot_commands": 0.007051504575289213, "deviation-center-line": 0.14656650223191586, "driven_lanedir_consec": 0.6423572123179142, "sim_compute_sim_state": 0.021410373541025016, "sim_compute_performance-ego": 0.005674515015039689, "sim_compute_robot_state-ego": 0.009113171161749424, "sim_compute_robot_state-npc0": 0.011034311392368415, "sim_compute_robot_state-npc1": 0.010932714511186648, "sim_compute_robot_state-npc2": 0.010725308687258989, "sim_compute_robot_state-npc3": 0.01375894668774727, "sim_compute_robot_state-parked0": 0.011245134549263196}, "ETHZ_autolab_technical_track-sc3-0-ego": {"driven_any": 2.491426417931044, "get_ui_image": 0.07885761428297612, "step_physics": 0.16976043843386465, "survival_time": 5.699999999999988, "driven_lanedir": 0.5314394847356438, "get_state_dump": 0.03895464068964908, "sim_render-ego": 0.010621223533362673, "in-drivable-lane": 4.04999999999999, "agent_compute-ego": 0.04660642147064209, "deviation-heading": 0.8355947377463032, "complete-iteration": 0.4533424691150063, "set_robot_commands": 0.007842385978029486, "deviation-center-line": 0.11963014860969592, "driven_lanedir_consec": 0.5314394847356438, "sim_compute_sim_state": 0.020810698208056, "sim_compute_performance-ego": 0.007119877296581603, "sim_compute_robot_state-ego": 0.013094736818681684, "sim_compute_robot_state-npc0": 0.01172754639073422, "sim_compute_robot_state-npc1": 0.011821711272524116, "sim_compute_robot_state-npc2": 0.011645896392956117, "sim_compute_robot_state-npc3": 0.011458309073197214, "sim_compute_robot_state-parked0": 0.012680628843474808}}
set_robot_commands_max0.007842385978029486
set_robot_commands_mean0.0070210073721004174
set_robot_commands_median0.0068514893286204265
set_robot_commands_min0.00653866485313133
sim_compute_performance-ego_max0.007119877296581603
sim_compute_performance-ego_mean0.006293616960779644
sim_compute_performance-ego_median0.006190037765748643
sim_compute_performance-ego_min0.005674515015039689
sim_compute_robot_state-ego_max0.013094736818681684
sim_compute_robot_state-ego_mean0.01072325562769717
sim_compute_robot_state-ego_median0.010342557265178784
sim_compute_robot_state-ego_min0.009113171161749424
sim_compute_robot_state-npc0_max0.01172754639073422
sim_compute_robot_state-npc0_mean0.01108446576767576
sim_compute_robot_state-npc0_median0.011112914805398708
sim_compute_robot_state-npc0_min0.010384487069171408
sim_compute_robot_state-npc1_max0.011821711272524116
sim_compute_robot_state-npc1_mean0.011186816843055244
sim_compute_robot_state-npc1_median0.011288799451626945
sim_compute_robot_state-npc1_min0.010347957196442978
sim_compute_robot_state-npc2_max0.011645896392956117
sim_compute_robot_state-npc2_mean0.010741080956150752
sim_compute_robot_state-npc2_median0.010703963431877288
sim_compute_robot_state-npc2_min0.009910500567892324
sim_compute_robot_state-npc3_max0.01375894668774727
sim_compute_robot_state-npc3_mean0.011677679577421038
sim_compute_robot_state-npc3_median0.01126837312129506
sim_compute_robot_state-npc3_min0.010415025379346764
sim_compute_robot_state-parked0_max0.012680628843474808
sim_compute_robot_state-parked0_mean0.011394502698881256
sim_compute_robot_state-parked0_median0.011237280660884672
sim_compute_robot_state-parked0_min0.010422820630280868
sim_compute_sim_state_max0.021410373541025016
sim_compute_sim_state_mean0.019680605541817532
sim_compute_sim_state_median0.020040872617539375
sim_compute_sim_state_min0.017230303391166355
sim_render-ego_max0.010621223533362673
sim_render-ego_mean0.009414857384180644
sim_render-ego_median0.00904497662604143
sim_render-ego_min0.008948252751277043
simulation-passed1
step_physics_max0.19733271391495413
step_physics_mean0.17880870887042966
step_physics_median0.17952980342199462
step_physics_min0.1588425147227752
survival_time_max5.699999999999988
survival_time_mean2.5374999999999974
survival_time_min1.1500000000000004
No reset possible
360849936Bea Baselines 🐤template-tensorflowaido5-LFV_multi-sim-validationLFVmultibodyv-simerrornoreg05-9e3e48edb8ad-10:01:32
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 49, in check_compatible_protocol
    raise IncompatibleProtocol(
zuper_nodes.compatibility.IncompatibleProtocol: For input "observations", cannot use type v2 as v21
│         k: observations
│        v1: dataclass aido_schemas.schemas.DB20ObservationsPlusState
│             field    camera : dataclass aido_schemas.protocol_simulator.JPGImage
│                                field jpg_data : bytes
│                                       __doc__
│                                                         An image in JPG format.
│ 
│                                                         jpg_data
│             field  odometry : dataclass aido_schemas.schemas.DB20Odometry
│                                field resolution_rad : float
│                                field  axis_left_rad : float
│                                field axis_right_rad : float
│             field your_name : str
│             field     state : dataclass aido_schemas.schemas.DTSimState
│                                field t_effective : float
│                                field  duckiebots : Dict[str,DTSimRobotInfo]
│             field  map_data : str
│        v2: dataclass aido_schemas.schemas.DB20Observations
│             field   camera : dataclass aido_schemas.protocol_simulator.JPGImage
│                               field jpg_data : bytes
│                                      __doc__
│                                                        An image in JPG format.
│ 
│                                                        jpg_data
│             field odometry : dataclass aido_schemas.schemas.DB20Odometry
│                               field resolution_rad : float
│                               field  axis_left_rad : float
│                               field axis_right_rad : float
│         r: CanBeUsed
│            │ result: False
│            │ why:
│            │ |Type <class 'aido_schemas.schemas.DB20ObservationsPlusState'>
│            │ | requires field "your_name"
│            │ |  of type <class 'str'>
│            │ |  but <class 'aido_schemas.schemas.DB20Observations'> does not have it.
│            │ M: Matches()
│            │ matches: {}
│ p1_inputs: Dict[str,type][4]
│            │ seed: int
│            │ get_commands:
│            │ dataclass aido_schemas.protocol_agent.GetCommands
│            │  field at_time : float
│            │ observations:
│            │ dataclass aido_schemas.schemas.DB20ObservationsPlusState
│            │  field    camera : dataclass aido_schemas.protocol_simulator.JPGImage
│            │                     field jpg_data : bytes
│            │                            __doc__
│            │                                              An image in JPG format.
│            │
│            │                                              jpg_data
│            │  field  odometry : dataclass aido_schemas.schemas.DB20Odometry
│            │                     field resolution_rad : float
│            │                     field  axis_left_rad : float
│            │                     field axis_right_rad : float
│            │  field your_name : str
│            │  field     state : dataclass aido_schemas.schemas.DTSimState
│            │                     field t_effective : float
│            │                     field  duckiebots : Dict[str,DTSimRobotInfo]
│            │  field  map_data : str
│            │ episode_start:
│            │ dataclass aido_schemas.protocol_agent.EpisodeStart
│            │  field episode_name : str
│            │             __doc__    Marker for the start of an episode.
│ p2_inputs: dict[4]
│            │ observations:
│            │ dataclass aido_schemas.schemas.DB20Observations
│            │  field   camera : dataclass aido_schemas.protocol_simulator.JPGImage
│            │                    field jpg_data : bytes
│            │                           __doc__
│            │                                             An image in JPG format.
│            │
│            │                                             jpg_data
│            │  field odometry : dataclass aido_schemas.schemas.DB20Odometry
│            │                    field resolution_rad : float
│            │                    field  axis_left_rad : float
│            │                    field axis_right_rad : float
│            │ seed: int
│            │ get_commands:
│            │ dataclass aido_schemas.protocol_agent.GetCommands
│            │  field at_time : float
│            │ episode_start:
│            │ dataclass aido_schemas.protocol_agent.EpisodeStart
│            │  field episode_name : str
│            │             __doc__    Marker for the start of an episode.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "experiment_manager.py", line 165, in main
    robot_ci._get_node_protocol(timeout=config.timeout_initialization)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 120, in _get_node_protocol
    check_compatible_protocol(self.node_protocol, self.expect_protocol)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes/compatibility.py", line 67, in check_compatible_protocol
    raise IncompatibleProtocol(msg, p1=p1, p2=p2) from e
zuper_nodes.compatibility.IncompatibleProtocol: Cannot say that p1 is a sub-protocol of p2
│ p1: InteractionProtocol
│     │ description: Particularization for Duckiebot1 observations and commands.
│     │ inputs:
│     │ Dict[str,type][4]
│     │ │ seed: int
│     │ │ get_commands:
│     │ │ dataclass aido_schemas.protocol_agent.GetCommands
│     │ │  field at_time : float
│     │ │ observations:
│     │ │ dataclass aido_schemas.schemas.DB20ObservationsPlusState
│     │ │  field    camera : dataclass aido_schemas.protocol_simulator.JPGImage
│     │ │                     field jpg_data : bytes
│     │ │                            __doc__
│     │ │                                              An image in JPG format.
│     │ │
│     │ │                                              jpg_data
│     │ │  field  odometry : dataclass aido_schemas.schemas.DB20Odometry
│     │ │                     field resolution_rad : float
│     │ │                     field  axis_left_rad : float
│     │ │                     field axis_right_rad : float
│     │ │  field your_name : str
│     │ │  field     state : dataclass aido_schemas.schemas.DTSimState
│     │ │                     field t_effective : float
│     │ │                     field  duckiebots : Dict[str,DTSimRobotInfo]
│     │ │  field  map_data : str
│     │ │ episode_start:
│     │ │ dataclass aido_schemas.protocol_agent.EpisodeStart
│     │ │  field episode_name : str
│     │ │             __doc__    Marker for the start of an episode.
│     │ outputs:
│     │ Dict[str,type][1]
│     │ │ commands:
│     │ │ dataclass aido_schemas.schemas.Duckiebot1Commands
│     │ │  field wheels : dataclass aido_schemas.schemas.PWMCommands
│     │ │                  field  motor_left : float
│     │ │                  field motor_right : float
│     │ │                            __doc__
│     │ │                                              PWM commands are floats between -1 and 1.
│     │ │  field   LEDS : dataclass aido_schemas.schemas.LEDSCommands
│     │ │                  field      center : dataclass aido_schemas.schemas.RGB
│     │ │                                       field       r : float
│     │ │                                       field       g : float
│     │ │                                       field       b : float
│     │ │                                             __doc__    Values between 0, 1.
│     │ │                  field  front_left : dataclass aido_schemas.schemas.RGB
│     │ │                                       field       r : float
│     │ │                                       field       g : float
│     │ │                                       field       b : float
│     │ │                                             __doc__    Values between 0, 1.
│     │ │                  field front_right : dataclass aido_schemas.schemas.RGB
│     │ │                                       field       r : float
│     │ │                                       field       g : float
│     │ │                                       field       b : float
│     │ │                                             __doc__    Values between 0, 1.
│     │ │                  field   back_left : dataclass aido_schemas.schemas.RGB
│     │ │                                       field       r : float
│     │ │                                       field       g : float
│     │ │                                       field       b : float
│     │ │                                             __doc__    Values between 0, 1.
│     │ │                  field  back_right : dataclass aido_schemas.schemas.RGB
│     │ │                                       field       r : float
│     │ │                                       field       g : float
│     │ │                                       field       b : float
│     │ │                                             __doc__    Values between 0, 1.
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)
│ p2: InteractionProtocol
│     │ description: Particularization for DB20 observations and commands.
│     │ inputs:
│     │ dict[4]
│     │ │ observations:
│     │ │ dataclass aido_schemas.schemas.DB20Observations
│     │ │  field   camera : dataclass aido_schemas.protocol_simulator.JPGImage
│     │ │                    field jpg_data : bytes
│     │ │                           __doc__
│     │ │                                             An image in JPG format.
│     │ │
│     │ │                                             jpg_data
│     │ │  field odometry : dataclass aido_schemas.schemas.DB20Odometry
│     │ │                    field resolution_rad : float
│     │ │                    field  axis_left_rad : float
│     │ │                    field axis_right_rad : float
│     │ │ seed: int
│     │ │ get_commands:
│     │ │ dataclass aido_schemas.protocol_agent.GetCommands
│     │ │  field at_time : float
│     │ │ episode_start:
│     │ │ dataclass aido_schemas.protocol_agent.EpisodeStart
│     │ │  field episode_name : str
│     │ │             __doc__    Marker for the start of an episode.
│     │ outputs:
│     │ dict[1]
│     │ │ commands:
│     │ │ dataclass aido_schemas.schemas.DB20Commands
│     │ │  field wheels : dataclass aido_schemas.schemas.PWMCommands
│     │ │                  field  motor_left : float
│     │ │                  field motor_right : float
│     │ │                            __doc__
│     │ │                                              PWM commands are floats between -1 and 1.
│     │ │  field   LEDS : dataclass aido_schemas.schemas.LEDSCommands
│     │ │                  field      center : dataclass aido_schemas.schemas.RGB
│     │ │                                       field       r : float
│     │ │                                       field       g : float
│     │ │                                       field       b : float
│     │ │                                             __doc__    Values between 0, 1.
│     │ │                  field  front_left : dataclass aido_schemas.schemas.RGB
│     │ │                                       field       r : float
│     │ │                                       field       g : float
│     │ │                                       field       b : float
│     │ │                                             __doc__    Values between 0, 1.
│     │ │                  field front_right : dataclass aido_schemas.schemas.RGB
│     │ │                                       field       r : float
│     │ │                                       field       g : float
│     │ │                                       field       b : float
│     │ │                                             __doc__    Values between 0, 1.
│     │ │                  field   back_left : dataclass aido_schemas.schemas.RGB
│     │ │                                       field       r : float
│     │ │                                       field       g : float
│     │ │                                       field       b : float
│     │ │                                             __doc__    Values between 0, 1.
│     │ │                  field  back_right : dataclass aido_schemas.schemas.RGB
│     │ │                                       field       r : float
│     │ │                                       field       g : float
│     │ │                                       field       b : float
│     │ │                                             __doc__    Values between 0, 1.
│     │ language: (in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 678, in scoring_context
    yield cie
  File "experiment_manager.py", line 638, in go
    wrap(cie)
  File "experiment_manager.py", line 623, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 294, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible