Duckietown Challenges Home Challenges Submissions

Evaluator 1589

ID1589
evaluatorip-172-31-43-40-10498
ownerI don't have one 😀
machineip-172-31-43-40
processip-172-31-43-40-10498
versiond-c:5.0.7;d-c-r:5.0.10;d-s:5.0.3
first heard
last heard
statusinactive
# evaluating
# success20 24484
# timeout
# failed7 24428
# error21 24368
# aborted
# host-error6 24378
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
251154390Jeick Hincapie Barrera 🇨🇴challenge-aido_LF-template-ros - Template solution using ROSaido3-LF-sim-testingstep1-simulationsuccessnoip-172-31-43-40-104980:12:15
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.4126113388371666
survival_time_median2.6999999999999984
deviation-center-line_median0.1187177598523136
in-drivable-lane_median0.9499999999999976


other stats
agent_compute-ego_max0.023237253228823345
agent_compute-ego_mean0.01996112037478808
agent_compute-ego_median0.020345296615209337
agent_compute-ego_min0.015057609631465033
deviation-center-line_max0.2616199012603187
deviation-center-line_mean0.12663752615997417
deviation-center-line_min0.07143676193760398
deviation-heading_max1.4563581054906
deviation-heading_mean0.5551840294909584
deviation-heading_median0.55940705176748
deviation-heading_min0.2596285487829773
driven_any_max1.6839250134630204
driven_any_mean0.7327294413370019
driven_any_median0.7309130538792374
driven_any_min0.2364518949491652
driven_lanedir_consec_max0.9416535020294268
driven_lanedir_consec_mean0.4469104002974222
driven_lanedir_consec_min0.2221238695940002
driven_lanedir_max0.9416535020294268
driven_lanedir_mean0.4469104002974222
driven_lanedir_median0.4126113388371666
driven_lanedir_min0.2221238695940002
in-drivable-lane_max2.1499999999999924
in-drivable-lane_mean0.8233333333333307
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.9226822529128632, "sim_physics": 0.0987465885323538, "survival_time": 3.5499999999999954, "driven_lanedir": 0.36592094708219225, "sim_render-ego": 0.012062804799684336, "in-drivable-lane": 1.8999999999999944, "agent_compute-ego": 0.02298066985439247, "deviation-heading": 0.2596285487829773, "set_robot_commands": 0.009444371075697348, "deviation-center-line": 0.16987818035809085, "driven_lanedir_consec": 0.36592094708219225, "sim_compute_sim_state": 0.005149844666601906, "sim_compute_performance-ego": 0.007894687249626912, "sim_compute_robot_state-ego": 0.009243830828599527}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.7993934895811183, "sim_physics": 0.10272918144861858, "survival_time": 2.9999999999999973, "driven_lanedir": 0.6632135184572033, "sim_render-ego": 0.012142761548360187, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.022359271844228108, "deviation-heading": 0.5809378480871985, "set_robot_commands": 0.009083799521128337, "deviation-center-line": 0.11667593935453972, "driven_lanedir_consec": 0.6632135184572033, "sim_compute_sim_state": 0.005052721500396729, "sim_compute_performance-ego": 0.007041752338409424, "sim_compute_robot_state-ego": 0.008891606330871582}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.2364518949491652, "sim_physics": 0.11023495414040306, "survival_time": 1.1000000000000003, "driven_lanedir": 0.2221238695940002, "sim_render-ego": 0.011499318209561434, "in-drivable-lane": 0, "agent_compute-ego": 0.022007400339299984, "deviation-heading": 0.34882560565471876, "set_robot_commands": 0.008479270068081942, "deviation-center-line": 0.11316665887359229, "driven_lanedir_consec": 0.2221238695940002, "sim_compute_sim_state": 0.004476525566794656, "sim_compute_performance-ego": 0.006060546094721014, "sim_compute_robot_state-ego": 0.00823266939683394}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.5946812776947944, "sim_physics": 0.106630211075147, "survival_time": 2.3999999999999995, "driven_lanedir": 0.5792955579972934, "sim_render-ego": 0.012498473127683004, "in-drivable-lane": 0, "agent_compute-ego": 0.023237253228823345, "deviation-heading": 0.568524016017798, "set_robot_commands": 0.008673002322514852, "deviation-center-line": 0.15908349194402024, "driven_lanedir_consec": 0.5792955579972934, "sim_compute_sim_state": 0.0048326800266901655, "sim_compute_performance-ego": 0.007622867822647095, "sim_compute_robot_state-ego": 0.008875747521718344}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.6091462024047467, "sim_physics": 0.09451397087263023, "survival_time": 2.3, "driven_lanedir": 0.2919817227673325, "sim_render-ego": 0.011536593022553818, "in-drivable-lane": 0.9499999999999992, "agent_compute-ego": 0.023237005524013355, "deviation-heading": 0.33908617460335944, "set_robot_commands": 0.008511138998943827, "deviation-center-line": 0.1011892720306074, "driven_lanedir_consec": 0.2919817227673325, "sim_compute_sim_state": 0.004544942275337551, "sim_compute_performance-ego": 0.006541096645852794, "sim_compute_robot_state-ego": 0.008200365564097528}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 0.3217723172279551, "sim_physics": 0.08123915394147237, "survival_time": 1.2000000000000004, "driven_lanedir": 0.259671941523266, "sim_render-ego": 0.00917587677637736, "in-drivable-lane": 0.05000000000000005, "agent_compute-ego": 0.017471303542455036, "deviation-heading": 0.55940705176748, "set_robot_commands": 0.006178170442581177, "deviation-center-line": 0.07983398841276981, "driven_lanedir_consec": 0.259671941523266, "sim_compute_sim_state": 0.003778090079625448, "sim_compute_performance-ego": 0.0048799216747283936, "sim_compute_robot_state-ego": 0.006724029779434204}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 0.9218874635466092, "sim_physics": 0.0878674892278818, "survival_time": 3.899999999999994, "driven_lanedir": 0.5450627289344152, "sim_render-ego": 0.010140281457167406, "in-drivable-lane": 1.2999999999999954, "agent_compute-ego": 0.020345296615209337, "deviation-heading": 0.6586708623908945, "set_robot_commands": 0.007991640995710324, "deviation-center-line": 0.13114121213460145, "driven_lanedir_consec": 0.5450627289344152, "sim_compute_sim_state": 0.004462303259433844, "sim_compute_performance-ego": 0.005736999022654998, "sim_compute_robot_state-ego": 0.00779117070711576}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 0.8035998912048737, "sim_physics": 0.0776476434298924, "survival_time": 2.799999999999998, "driven_lanedir": 0.4126113388371666, "sim_render-ego": 0.00944221019744873, "in-drivable-lane": 0.9999999999999972, "agent_compute-ego": 0.018745856625693183, "deviation-heading": 0.6465417997696937, "set_robot_commands": 0.006655433348246983, "deviation-center-line": 0.119844131874105, "driven_lanedir_consec": 0.4126113388371666, "sim_compute_sim_state": 0.0037874707153865263, "sim_compute_performance-ego": 0.005336467708860125, "sim_compute_robot_state-ego": 0.006962303604398455}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 1.0820250408860437, "sim_physics": 0.07665973422171056, "survival_time": 4.3499999999999925, "driven_lanedir": 0.4507633352947824, "sim_render-ego": 0.009690007944216674, "in-drivable-lane": 2.1499999999999924, "agent_compute-ego": 0.022410494157637673, "deviation-heading": 0.613751936282656, "set_robot_commands": 0.007374122225005051, "deviation-center-line": 0.1187177598523136, "driven_lanedir_consec": 0.4507633352947824, "sim_compute_sim_state": 0.004195703857246487, "sim_compute_performance-ego": 0.005437198726610206, "sim_compute_robot_state-ego": 0.006748457064573792}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 1.6839250134630204, "sim_physics": 0.07940614440224388, "survival_time": 5.4999999999999885, "driven_lanedir": 0.9416535020294268, "sim_render-ego": 0.009421667185696689, "in-drivable-lane": 1.7999999999999936, "agent_compute-ego": 0.019046037847345525, "deviation-heading": 1.4563581054906, "set_robot_commands": 0.006567768617109819, "deviation-center-line": 0.2616199012603187, "driven_lanedir_consec": 0.9416535020294268, "sim_compute_sim_state": 0.004058426076715643, "sim_compute_performance-ego": 0.005414496768604626, "sim_compute_robot_state-ego": 0.006938574530861595}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 0.26677752846248903, "sim_physics": 0.09250099460283916, "survival_time": 1.2000000000000004, "driven_lanedir": 0.25235480657068055, "sim_render-ego": 0.010140597820281982, "in-drivable-lane": 0, "agent_compute-ego": 0.021265476942062374, "deviation-heading": 0.2735028974766809, "set_robot_commands": 0.007510453462600708, "deviation-center-line": 0.10630271043038582, "driven_lanedir_consec": 0.25235480657068055, "sim_compute_sim_state": 0.0042486488819122314, "sim_compute_performance-ego": 0.005909889936447144, "sim_compute_robot_state-ego": 0.00759089986483256}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 1.0389325864387629, "sim_physics": 0.06972543914596756, "survival_time": 3.8499999999999943, "driven_lanedir": 0.4755729783153157, "sim_render-ego": 0.009104270439643364, "in-drivable-lane": 1.7499999999999938, "agent_compute-ego": 0.018038656804468724, "deviation-heading": 0.5216992361385433, "set_robot_commands": 0.006197805528516893, "deviation-center-line": 0.10517150645857888, "driven_lanedir_consec": 0.4755729783153157, "sim_compute_sim_state": 0.0038694685155695138, "sim_compute_performance-ego": 0.005179337092808315, "sim_compute_robot_state-ego": 0.006746264247151164}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 0.6835794874956865, "sim_physics": 0.06695712089538575, "survival_time": 2.499999999999999, "driven_lanedir": 0.6100305079584653, "sim_render-ego": 0.009170331954956056, "in-drivable-lane": 0.14999999999999947, "agent_compute-ego": 0.01660872459411621, "deviation-heading": 0.4739455376152245, "set_robot_commands": 0.006392812728881836, "deviation-center-line": 0.12143860235134882, "driven_lanedir_consec": 0.6100305079584653, "sim_compute_sim_state": 0.0035709238052368166, "sim_compute_performance-ego": 0.005188465118408203, "sim_compute_robot_state-ego": 0.006484522819519043}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 0.29517411990766207, "sim_physics": 0.057311378992520846, "survival_time": 1.3000000000000005, "driven_lanedir": 0.25251967754950355, "sim_render-ego": 0.0073831539887648364, "in-drivable-lane": 0.05000000000000005, "agent_compute-ego": 0.015057609631465033, "deviation-heading": 0.466101326638835, "set_robot_commands": 0.0052050627194918115, "deviation-center-line": 0.07143676193760398, "driven_lanedir_consec": 0.25251967754950355, "sim_compute_sim_state": 0.002978315720191369, "sim_compute_performance-ego": 0.004050539090083196, "sim_compute_robot_state-ego": 0.005144192622258113}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 0.7309130538792374, "sim_physics": 0.0706709314275671, "survival_time": 2.6999999999999984, "driven_lanedir": 0.38087957155028906, "sim_render-ego": 0.007931678383438676, "in-drivable-lane": 0.9499999999999976, "agent_compute-ego": 0.016605748070610896, "deviation-heading": 0.5607794956477159, "set_robot_commands": 0.005836628101490162, "deviation-center-line": 0.12406277512673572, "driven_lanedir_consec": 0.38087957155028906, "sim_compute_sim_state": 0.0034197568893432617, "sim_compute_performance-ego": 0.004365611959386755, "sim_compute_robot_state-ego": 0.005849855917471426}}
set_robot_commands_max0.009444371075697348
set_robot_commands_mean0.007340098677066739
set_robot_commands_median0.007374122225005051
set_robot_commands_min0.0052050627194918115
sim_compute_performance-ego_max0.007894687249626912
sim_compute_performance-ego_mean0.0057773251499899465
sim_compute_performance-ego_median0.005437198726610206
sim_compute_performance-ego_min0.004050539090083196
sim_compute_robot_state-ego_max0.009243830828599527
sim_compute_robot_state-ego_mean0.007361632719982469
sim_compute_robot_state-ego_median0.006962303604398455
sim_compute_robot_state-ego_min0.005144192622258113
sim_compute_sim_state_max0.005149844666601906
sim_compute_sim_state_mean0.004161721455765477
sim_compute_sim_state_median0.004195703857246487
sim_compute_sim_state_min0.002978315720191369
sim_physics_max0.11023495414040306
sim_physics_mean0.08485606242377561
sim_physics_median0.08123915394147237
sim_physics_min0.057311378992520846
sim_render-ego_max0.012498473127683004
sim_render-ego_mean0.010089335123722305
sim_render-ego_median0.009690007944216674
sim_render-ego_min0.0073831539887648364
simulation-passed1
survival_time_max5.4999999999999885
survival_time_mean2.7766666666666637
survival_time_min1.1000000000000003
No reset possible
251074386Soroush Saryazdi 🇨🇦bluejayaido3-LF-sim-validationstep1-simulationerrornoip-172-31-43-40-104980:05:46
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
251044382Soroush Saryazdi 🇨🇦bluejayaido3-LFV-sim-testingstep1-simulationfailednoip-172-31-43-40-104980:05:20
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 177, in read_one
    nickname=self.nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 258, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 296, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "agent".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "experiment_manager.py", line 299, in run_episode
    r: MsgReceived = agent.write_topic_and_expect('get_commands', expect='commands')
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 96, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 239, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/agent-out after 73 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/duckietown_challenges/cie_concrete.py", line 675, in scoring_context
    yield cie
  File "experiment_manager.py", line 444, in <module>
    wrap(cie)
  File "experiment_manager.py", line 432, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 136, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 303, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
250754367Jeick Hincapie Barrera 🇨🇴challenge-aido_LF-template-tensorflowaido3-LF-sim-validationstep1-simulationsuccessnoip-172-31-43-40-104980:06:51
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.2368442691241024
survival_time_median1.800000000000001
deviation-center-line_median0.0856185707636209
in-drivable-lane_median0.05000000000000005


other stats
agent_compute-ego_max0.033522333417619975
agent_compute-ego_mean0.02777797692807984
agent_compute-ego_median0.02712776464059812
agent_compute-ego_min0.02478214233152328
deviation-center-line_max0.08821763628729837
deviation-center-line_mean0.08221872401668547
deviation-center-line_min0.07447096127621734
deviation-heading_max0.7283948566777063
deviation-heading_mean0.5665181424854218
deviation-heading_median0.560496636612824
deviation-heading_min0.41074597681365943
driven_any_max1.2205768978879286
driven_any_mean0.5680873404526502
driven_any_median0.49608797043752534
driven_any_min0.19886916371752025
driven_lanedir_consec_max0.4260056079541781
driven_lanedir_consec_mean0.24301056143947108
driven_lanedir_consec_min0.10017190954408496
driven_lanedir_max0.4260056079541781
driven_lanedir_mean0.24301056143947108
driven_lanedir_median0.2368442691241024
driven_lanedir_min0.10017190954408496
in-drivable-lane_max4.299999999999988
in-drivable-lane_mean1.1199999999999974
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.19886916371752025, "sim_physics": 0.13718682243710473, "survival_time": 1.0500000000000005, "driven_lanedir": 0.10017190954408496, "sim_render-ego": 0.012915270669119698, "in-drivable-lane": 0, "agent_compute-ego": 0.033522333417619975, "deviation-heading": 0.7283948566777063, "set_robot_commands": 0.009422097887311662, "deviation-center-line": 0.07608603070367861, "driven_lanedir_consec": 0.10017190954408496, "sim_compute_sim_state": 0.006485371362595331, "sim_compute_performance-ego": 0.007819175720214844, "sim_compute_robot_state-ego": 0.00962988535563151}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 1.2205768978879286, "sim_physics": 0.09666709287450947, "survival_time": 5.449999999999989, "driven_lanedir": 0.2368442691241024, "sim_render-ego": 0.01222374461112766, "in-drivable-lane": 4.299999999999988, "agent_compute-ego": 0.02712776464059812, "deviation-heading": 0.41074597681365943, "set_robot_commands": 0.008495295813324255, "deviation-center-line": 0.0856185707636209, "driven_lanedir_consec": 0.2368442691241024, "sim_compute_sim_state": 0.0049361220193565435, "sim_compute_performance-ego": 0.006975854208709996, "sim_compute_robot_state-ego": 0.008639611235452354}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.5612662789257065, "sim_physics": 0.1052361528078715, "survival_time": 2.3999999999999995, "driven_lanedir": 0.15655801250673518, "sim_render-ego": 0.01209352413813273, "in-drivable-lane": 1.199999999999999, "agent_compute-ego": 0.027275284131368, "deviation-heading": 0.6919172834106756, "set_robot_commands": 0.008604218562444052, "deviation-center-line": 0.07447096127621734, "driven_lanedir_consec": 0.15655801250673518, "sim_compute_sim_state": 0.005316793918609619, "sim_compute_performance-ego": 0.007139722506205241, "sim_compute_robot_state-ego": 0.008532931407292684}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.3636363912945705, "sim_physics": 0.09567596066382623, "survival_time": 1.5500000000000007, "driven_lanedir": 0.2954730080682548, "sim_render-ego": 0.0122873475474696, "in-drivable-lane": 0.05000000000000005, "agent_compute-ego": 0.02478214233152328, "deviation-heading": 0.560496636612824, "set_robot_commands": 0.010001328683668568, "deviation-center-line": 0.08821763628729837, "driven_lanedir_consec": 0.2954730080682548, "sim_compute_sim_state": 0.004623789941110919, "sim_compute_performance-ego": 0.007008098786877048, "sim_compute_robot_state-ego": 0.009283950251917686}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.49608797043752534, "sim_physics": 0.09687831666734484, "survival_time": 1.800000000000001, "driven_lanedir": 0.4260056079541781, "sim_render-ego": 0.011974261866675485, "in-drivable-lane": 0.05000000000000005, "agent_compute-ego": 0.026182360119289823, "deviation-heading": 0.4410359589122433, "set_robot_commands": 0.009213990635342067, "deviation-center-line": 0.08670042105261214, "driven_lanedir_consec": 0.4260056079541781, "sim_compute_sim_state": 0.004708912637498643, "sim_compute_performance-ego": 0.006722384028964573, "sim_compute_robot_state-ego": 0.008663707309299044}}
set_robot_commands_max0.010001328683668568
set_robot_commands_mean0.00914738631641812
set_robot_commands_median0.009213990635342067
set_robot_commands_min0.008495295813324255
sim_compute_performance-ego_max0.007819175720214844
sim_compute_performance-ego_mean0.007133047050194341
sim_compute_performance-ego_median0.007008098786877048
sim_compute_performance-ego_min0.006722384028964573
sim_compute_robot_state-ego_max0.00962988535563151
sim_compute_robot_state-ego_mean0.008950017111918656
sim_compute_robot_state-ego_median0.008663707309299044
sim_compute_robot_state-ego_min0.008532931407292684
sim_compute_sim_state_max0.006485371362595331
sim_compute_sim_state_mean0.005214197975834211
sim_compute_sim_state_median0.0049361220193565435
sim_compute_sim_state_min0.004623789941110919
sim_physics_max0.13718682243710473
sim_physics_mean0.10632886909013135
sim_physics_median0.09687831666734484
sim_physics_min0.09567596066382623
sim_render-ego_max0.012915270669119698
sim_render-ego_mean0.012298829766505034
sim_render-ego_median0.01222374461112766
sim_render-ego_min0.011974261866675485
simulation-passed1
survival_time_max5.449999999999989
survival_time_mean2.449999999999998
survival_time_min1.0500000000000005
No reset possible
250254357Soroush Saryazdi 🇨🇦bluejayaido3-LF-sim-testingstep1-simulationerrornoip-172-31-43-40-104980:02:43
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
249824148Andrea Censi 🇨🇭rotationaido3-LFVI-sim-testingstep1-simulationhost-errornoip-172-31-43-40-104980:59:24
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 641, in get_cr
    wd, aws_config, copy_to_machine_cache=copy_to_machine_cache
  File "/project/src/duckietown_challenges_runner/runner.py", line 1301, in upload_files
    aws_config, toupload, copy_to_machine_cache=copy_to_machine_cache
  File "/project/src/duckietown_challenges_runner/runner.py", line 1555, in upload
    sha256hex = compute_sha256hex(realfile)
  File "/project/src/duckietown_challenges_runner/runner.py", line 1611, in compute_sha256hex
    res: bytes = subprocess.check_output(cmd)
  File "/usr/lib/python3.6/subprocess.py", line 356, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['shasum', '-a', '256', '/tmp/duckietown/DT18/evaluator/executions/aido3-LFVI-sim-testing/submission4148/step1-simulation-ip-172-31-43-40-10498-job24982/logs/simulator/stderr.log']' returned non-zero exit status 1.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
249774166Tanli Chou 🇹🇼challenge-aido_LF-template-randomaido3-LFV-sim-validationstep1-simulationerrornoip-172-31-43-40-104980:01:51
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
249724170Bhairav Mehtachallenge-aido_LF-template-pytorchaido3-LF-sim-testingstep1-simulationfailednoip-172-31-43-40-104980:00:57
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "experiment_manager.py", line 298, in run_episode
    agent.write_topic_and_expect_zero('observations', recv.data.observations)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 105, in write_topic_and_expect_zero
    nickname=self.nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 258, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 286, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "agent" aborted:

error in agent |Exception while handling a message on topic "observations".
               |
               || Traceback (most recent call last):
               ||   File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper.py", line 277, in loop
               ||     handle_message_node(parsed, receiver0, context0)
               ||   File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper.py", line 431, in handle_message_node
               ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
               ||   File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/utils.py", line 19, in call_if_fun_exists
               ||     f(**kwargs)
               ||   File "solution.py", line 36, in on_received_observations
               ||     self.current_image = self.preprocessor.preprocess(obs)
               ||   File "/workspace/wrappers.py", line 9, in preprocess
               ||     from scipy.misc import imresize
               || ImportError: cannot import name 'imresize' from 'scipy.misc' (/usr/local/lib/python3.7/site-packages/scipy/misc/__init__.py)
               ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/duckietown_challenges/cie_concrete.py", line 675, in scoring_context
    yield cie
  File "experiment_manager.py", line 444, in <module>
    wrap(cie)
  File "experiment_manager.py", line 432, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 136, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 303, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
249684176Bhairav Mehtachallenge-aido_LF-template-pytorchaido3-LFVI-sim-validationstep1-simulationfailednoip-172-31-43-40-104980:02:04
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "experiment_manager.py", line 298, in run_episode
    agent.write_topic_and_expect_zero('observations', recv.data.observations)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 105, in write_topic_and_expect_zero
    nickname=self.nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 258, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 286, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "agent" aborted:

error in agent |Exception while handling a message on topic "observations".
               |
               || Traceback (most recent call last):
               ||   File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper.py", line 277, in loop
               ||     handle_message_node(parsed, receiver0, context0)
               ||   File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper.py", line 431, in handle_message_node
               ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
               ||   File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/utils.py", line 19, in call_if_fun_exists
               ||     f(**kwargs)
               ||   File "solution.py", line 36, in on_received_observations
               ||     self.current_image = self.preprocessor.preprocess(obs)
               ||   File "/workspace/wrappers.py", line 9, in preprocess
               ||     from scipy.misc import imresize
               || ImportError: cannot import name 'imresize' from 'scipy.misc' (/usr/local/lib/python3.7/site-packages/scipy/misc/__init__.py)
               ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/duckietown_challenges/cie_concrete.py", line 675, in scoring_context
    yield cie
  File "experiment_manager.py", line 444, in <module>
    wrap(cie)
  File "experiment_manager.py", line 432, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 136, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 303, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
249664181Liam Paull 🇨🇦challenge-aido_LF-baseline-duckietownaido3-LFV-sim-testingstep1-simulationerrornoip-172-31-43-40-104980:00:31
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
249634182Liam Paull 🇨🇦challenge-aido_LF-baseline-duckietownaido3-LFV-sim-validationstep1-simulationerrornoip-172-31-43-40-104980:00:33
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
249584184Liam Paull 🇨🇦challenge-aido_LF-baseline-duckietownaido3-LFVI-sim-validationstep1-simulationerrornoip-172-31-43-40-104980:00:45
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
249534195Liam Paull 🇨🇦challenge-aido_LF-baseline-duckietownaido3-LF-sim-testingstep1-simulationerrornoip-172-31-43-40-104980:00:47
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
249344212Bhairav Mehtachallenge-aido_LF-template-pytorchaido3-LFVI-sim-validationstep1-simulationfailednoip-172-31-43-40-104980:05:26
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "experiment_manager.py", line 298, in run_episode
    agent.write_topic_and_expect_zero('observations', recv.data.observations)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 105, in write_topic_and_expect_zero
    nickname=self.nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 258, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 296, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "agent".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/duckietown_challenges/cie_concrete.py", line 675, in scoring_context
    yield cie
  File "experiment_manager.py", line 444, in <module>
    wrap(cie)
  File "experiment_manager.py", line 432, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 136, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 303, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
249264222Manfred DiazBaseline-IL-logs-tensorflowaido3-LF-sim-testingstep1-simulationfailednoip-172-31-43-40-104980:00:59
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "experiment_manager.py", line 299, in run_episode
    r: MsgReceived = agent.write_topic_and_expect('get_commands', expect='commands')
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 96, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 177, in read_one
    nickname=self.nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 258, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 286, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "agent" aborted:

error in agent |Exception while handling a message on topic "get_commands".
               |
               || Traceback (most recent call last):
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1334, in _do_call
               ||     return fn(*args)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
               ||     options, feed_dict, fetch_list, target_list, run_metadata)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
               ||     run_metadata)
               || tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value prefix/ConvNet/fc_layer_2/kernel
               || 	 [[{{node prefix/ConvNet/fc_layer_2/kernel/read}}]]
               ||
               || During handling of the above exception, another exception occurred:
               ||
               || Traceback (most recent call last):
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 277, in loop
               ||     handle_message_node(parsed, receiver0, context0)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 431, in handle_message_node
               ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/utils.py", line 19, in call_if_fun_exists
               ||     f(**kwargs)
               ||   File "imitation_agent.py", line 71, in on_received_get_commands
               ||     pwm_left, pwm_right = self.compute_action(self.config.current_image)
               ||   File "imitation_agent.py", line 60, in compute_action
               ||     x: observation
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 929, in run
               ||     run_metadata_ptr)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1152, in _run
               ||     feed_dict_tensor, options, run_metadata)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1328, in _do_run
               ||     run_metadata)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1348, in _do_call
               ||     raise type(e)(node_def, op, message)
               || tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value prefix/ConvNet/fc_layer_2/kernel
               || 	 [[node prefix/ConvNet/fc_layer_2/kernel/read (defined at /workspace/graph_utils.py:14) ]]
               ||
               || Caused by op 'prefix/ConvNet/fc_layer_2/kernel/read', defined at:
               ||   File "imitation_agent.py", line 136, in <module>
               ||     main()
               ||   File "imitation_agent.py", line 132, in main
               ||     wrap_direct(node=node, protocol=protocol)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/interface.py", line 22, in wrap_direct
               ||     run_loop(node, protocol, args)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 196, in run_loop
               ||     config=config)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 277, in loop
               ||     handle_message_node(parsed, receiver0, context0)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 431, in handle_message_node
               ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/utils.py", line 19, in call_if_fun_exists
               ||     f(**kwargs)
               ||   File "imitation_agent.py", line 71, in on_received_get_commands
               ||     pwm_left, pwm_right = self.compute_action(self.config.current_image)
               ||   File "imitation_agent.py", line 43, in compute_action
               ||     graph = load_graph(frozen_model_filename)
               ||   File "/workspace/graph_utils.py", line 14, in load_graph
               ||     tf.import_graph_def(graph_def, name="prefix")
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
               ||     return func(*args, **kwargs)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/importer.py", line 442, in import_graph_def
               ||     _ProcessNewOps(graph)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/importer.py", line 235, in _ProcessNewOps
               ||     for new_op in graph._add_new_tf_operations(compute_devices=False):  # pylint: disable=protected-access
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3433, in _add_new_tf_operations
               ||     for c_op in c_api_util.new_tf_operations(self)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3433, in <listcomp>
               ||     for c_op in c_api_util.new_tf_operations(self)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3325, in _create_op_from_tf_operation
               ||     ret = Operation(c_op, self)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 1801, in __init__
               ||     self._traceback = tf_stack.extract_stack()
               ||
               || FailedPreconditionError (see above for traceback): Attempting to use uninitialized value prefix/ConvNet/fc_layer_2/kernel
               || 	 [[node prefix/ConvNet/fc_layer_2/kernel/read (defined at /workspace/graph_utils.py:14) ]]
               ||
               ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/duckietown_challenges/cie_concrete.py", line 675, in scoring_context
    yield cie
  File "experiment_manager.py", line 444, in <module>
    wrap(cie)
  File "experiment_manager.py", line 432, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 136, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 303, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
249194228Manfred DiazBaseline-IL-logs-tensorflowaido3-LFVI-sim-validationstep1-simulationfailednoip-172-31-43-40-104980:02:33
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "experiment_manager.py", line 299, in run_episode
    r: MsgReceived = agent.write_topic_and_expect('get_commands', expect='commands')
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 96, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 177, in read_one
    nickname=self.nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 258, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 286, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "agent" aborted:

error in agent |Exception while handling a message on topic "get_commands".
               |
               || Traceback (most recent call last):
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1334, in _do_call
               ||     return fn(*args)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
               ||     options, feed_dict, fetch_list, target_list, run_metadata)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
               ||     run_metadata)
               || tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value prefix/ConvNet/fc_layer_2/bias
               || 	 [[{{node prefix/ConvNet/fc_layer_2/bias/read}}]]
               ||
               || During handling of the above exception, another exception occurred:
               ||
               || Traceback (most recent call last):
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 277, in loop
               ||     handle_message_node(parsed, receiver0, context0)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 431, in handle_message_node
               ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/utils.py", line 19, in call_if_fun_exists
               ||     f(**kwargs)
               ||   File "imitation_agent.py", line 71, in on_received_get_commands
               ||     pwm_left, pwm_right = self.compute_action(self.config.current_image)
               ||   File "imitation_agent.py", line 60, in compute_action
               ||     x: observation
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 929, in run
               ||     run_metadata_ptr)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1152, in _run
               ||     feed_dict_tensor, options, run_metadata)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1328, in _do_run
               ||     run_metadata)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1348, in _do_call
               ||     raise type(e)(node_def, op, message)
               || tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value prefix/ConvNet/fc_layer_2/bias
               || 	 [[node prefix/ConvNet/fc_layer_2/bias/read (defined at /workspace/graph_utils.py:14) ]]
               ||
               || Caused by op 'prefix/ConvNet/fc_layer_2/bias/read', defined at:
               ||   File "imitation_agent.py", line 136, in <module>
               ||     main()
               ||   File "imitation_agent.py", line 132, in main
               ||     wrap_direct(node=node, protocol=protocol)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/interface.py", line 22, in wrap_direct
               ||     run_loop(node, protocol, args)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 196, in run_loop
               ||     config=config)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 277, in loop
               ||     handle_message_node(parsed, receiver0, context0)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 431, in handle_message_node
               ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/utils.py", line 19, in call_if_fun_exists
               ||     f(**kwargs)
               ||   File "imitation_agent.py", line 71, in on_received_get_commands
               ||     pwm_left, pwm_right = self.compute_action(self.config.current_image)
               ||   File "imitation_agent.py", line 43, in compute_action
               ||     graph = load_graph(frozen_model_filename)
               ||   File "/workspace/graph_utils.py", line 14, in load_graph
               ||     tf.import_graph_def(graph_def, name="prefix")
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
               ||     return func(*args, **kwargs)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/importer.py", line 442, in import_graph_def
               ||     _ProcessNewOps(graph)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/importer.py", line 235, in _ProcessNewOps
               ||     for new_op in graph._add_new_tf_operations(compute_devices=False):  # pylint: disable=protected-access
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3433, in _add_new_tf_operations
               ||     for c_op in c_api_util.new_tf_operations(self)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3433, in <listcomp>
               ||     for c_op in c_api_util.new_tf_operations(self)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3325, in _create_op_from_tf_operation
               ||     ret = Operation(c_op, self)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 1801, in __init__
               ||     self._traceback = tf_stack.extract_stack()
               ||
               || FailedPreconditionError (see above for traceback): Attempting to use uninitialized value prefix/ConvNet/fc_layer_2/bias
               || 	 [[node prefix/ConvNet/fc_layer_2/bias/read (defined at /workspace/graph_utils.py:14) ]]
               ||
               ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/duckietown_challenges/cie_concrete.py", line 675, in scoring_context
    yield cie
  File "experiment_manager.py", line 444, in <module>
    wrap(cie)
  File "experiment_manager.py", line 432, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 136, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 303, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
249174235Andrea Censi 🇨🇭challenge-aido_LF-template-pytorchaido3-LFVI-sim-testingstep1-simulationerrornoip-172-31-43-40-104980:00:35
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
249134236Andrea Censi 🇨🇭challenge-aido_LF-template-pytorchaido3-LFVI-sim-validationstep1-simulationerrornoip-172-31-43-40-104980:01:44
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
249094240Manfred DiazBaseline-IL-logs-tensorflowaido3-LF-sim-validationstep1-simulationsuccessnoip-172-31-43-40-104980:16:55
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.034362774422512565
survival_time_median14.950000000000076
deviation-center-line_median0.8267657840223475
in-drivable-lane_median0


other stats
agent_compute-ego_max0.1015104873975118
agent_compute-ego_mean0.08858231401443481
agent_compute-ego_median0.08843233982721964
agent_compute-ego_min0.07947943290074666
deviation-center-line_max1.1089020709307842
deviation-center-line_mean0.7593289556437867
deviation-center-line_min0.31685496930609974
deviation-heading_max4.999484291668101
deviation-heading_mean2.6753246660585432
deviation-heading_median2.1626495929760305
deviation-heading_min0.7966571787719199
driven_any_max0.034756089148517647
driven_any_mean0.03330784795381577
driven_any_median0.034741540279710326
driven_any_min0.027558529781427116
driven_lanedir_consec_max0.03470214620791623
driven_lanedir_consec_mean0.032643903122578255
driven_lanedir_consec_min0.02603099086701155
driven_lanedir_max0.03470214620791623
driven_lanedir_mean0.032643903122578255
driven_lanedir_median0.034362774422512565
driven_lanedir_min0.02603099086701155
in-drivable-lane_max0
in-drivable-lane_mean0
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.034756089148517647, "sim_physics": 0.10656540075937906, "survival_time": 14.950000000000076, "driven_lanedir": 0.03456000889545052, "sim_render-ego": 0.011694986820220948, "in-drivable-lane": 0, "agent_compute-ego": 0.08375924825668335, "deviation-heading": 1.568987939257715, "set_robot_commands": 0.009067623615264891, "deviation-center-line": 1.1089020709307842, "driven_lanedir_consec": 0.03456000889545052, "sim_compute_sim_state": 0.0047521042823791505, "sim_compute_performance-ego": 0.007430625756581624, "sim_compute_robot_state-ego": 0.009363052050272625}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.034741540279710326, "sim_physics": 0.11172621011734007, "survival_time": 14.950000000000076, "driven_lanedir": 0.034362774422512565, "sim_render-ego": 0.011732758680979413, "in-drivable-lane": 0, "agent_compute-ego": 0.08843233982721964, "deviation-heading": 2.1626495929760305, "set_robot_commands": 0.009876292546590167, "deviation-center-line": 0.31685496930609974, "driven_lanedir_consec": 0.034362774422512565, "sim_compute_sim_state": 0.005027018388112386, "sim_compute_performance-ego": 0.007031254768371582, "sim_compute_robot_state-ego": 0.009034474690755207}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.03474154027971588, "sim_physics": 0.0998653252919515, "survival_time": 14.950000000000076, "driven_lanedir": 0.03356359522000041, "sim_render-ego": 0.011718597412109375, "in-drivable-lane": 0, "agent_compute-ego": 0.07947943290074666, "deviation-heading": 3.848844327618949, "set_robot_commands": 0.008813347816467285, "deviation-center-line": 0.9869532432869856, "driven_lanedir_consec": 0.03356359522000041, "sim_compute_sim_state": 0.004883678754170736, "sim_compute_performance-ego": 0.006822575728098551, "sim_compute_robot_state-ego": 0.008638630708058676}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.027558529781427116, "sim_physics": 0.12167116721471152, "survival_time": 14.950000000000076, "driven_lanedir": 0.02603099086701155, "sim_render-ego": 0.012408878008524577, "in-drivable-lane": 0, "agent_compute-ego": 0.1015104873975118, "deviation-heading": 4.999484291668101, "set_robot_commands": 0.010100393295288088, "deviation-center-line": 0.8267657840223475, "driven_lanedir_consec": 0.02603099086701155, "sim_compute_sim_state": 0.005291148026784261, "sim_compute_performance-ego": 0.007536858717600505, "sim_compute_robot_state-ego": 0.00964004119237264}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.03474154027970788, "sim_physics": 0.10717925469080608, "survival_time": 14.950000000000076, "driven_lanedir": 0.03470214620791623, "sim_render-ego": 0.012397832870483398, "in-drivable-lane": 0, "agent_compute-ego": 0.08973006169001262, "deviation-heading": 0.7966571787719199, "set_robot_commands": 0.009544971783955893, "deviation-center-line": 0.5571687106727162, "driven_lanedir_consec": 0.03470214620791623, "sim_compute_sim_state": 0.005303552945454915, "sim_compute_performance-ego": 0.0072330943743387855, "sim_compute_robot_state-ego": 0.008993504842122396}}
set_robot_commands_max0.010100393295288088
set_robot_commands_mean0.009480525811513266
set_robot_commands_median0.009544971783955893
set_robot_commands_min0.008813347816467285
sim_compute_performance-ego_max0.007536858717600505
sim_compute_performance-ego_mean0.007210881868998209
sim_compute_performance-ego_median0.0072330943743387855
sim_compute_performance-ego_min0.006822575728098551
sim_compute_robot_state-ego_max0.00964004119237264
sim_compute_robot_state-ego_mean0.009133940696716308
sim_compute_robot_state-ego_median0.009034474690755207
sim_compute_robot_state-ego_min0.008638630708058676
sim_compute_sim_state_max0.005303552945454915
sim_compute_sim_state_mean0.00505150047938029
sim_compute_sim_state_median0.005027018388112386
sim_compute_sim_state_min0.0047521042823791505
sim_physics_max0.12167116721471152
sim_physics_mean0.10940147161483764
sim_physics_median0.10717925469080608
sim_physics_min0.0998653252919515
sim_render-ego_max0.012408878008524577
sim_render-ego_mean0.01199061075846354
sim_render-ego_median0.011732758680979413
sim_render-ego_min0.011694986820220948
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean14.950000000000076
survival_time_min14.950000000000076
No reset possible
249054246Manfred Diazchallenge-aido_LF-template-tensorflow - Template solution using imitation learningaido3-LF-sim-validationstep1-simulationerrornoip-172-31-43-40-104980:02:09
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
249004249Manfred Diazchallenge-aido_LF-template-tensorflowaido3-LF-sim-validationstep1-simulationsuccessnoip-172-31-43-40-104980:06:43
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.3891331688028239
survival_time_median3.699999999999995
deviation-center-line_median0.15947327091615762
in-drivable-lane_median1.7499999999999971


other stats
agent_compute-ego_max0.04308545267259752
agent_compute-ego_mean0.02828316127650692
agent_compute-ego_median0.025754521952735053
agent_compute-ego_min0.02009568764613225
deviation-center-line_max0.469380609026831
deviation-center-line_mean0.1918704636183151
deviation-center-line_min0.06780723411796019
deviation-heading_max3.4772722656270423
deviation-heading_mean1.4923244674821308
deviation-heading_median1.1407295395512222
deviation-heading_min0.7390595710504796
driven_any_max2.158857000199323
driven_any_mean0.9237918064876144
driven_any_median0.9053666680722884
driven_any_min0.20584484287405633
driven_lanedir_consec_max1.4193283265977223
driven_lanedir_consec_mean0.4947342188620623
driven_lanedir_consec_min0.11923943865150877
driven_lanedir_max1.4193283265977223
driven_lanedir_mean0.4947342188620623
driven_lanedir_median0.3891331688028239
driven_lanedir_min0.11923943865150877
in-drivable-lane_max2.1499999999999924
in-drivable-lane_mean1.199999999999997
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.20584484287405633, "sim_physics": 0.09210938992707626, "survival_time": 1.1500000000000004, "driven_lanedir": 0.11923943865150877, "sim_render-ego": 0.010328562363334324, "in-drivable-lane": 0, "agent_compute-ego": 0.02554619830587636, "deviation-heading": 0.7554826182114563, "set_robot_commands": 0.007210845532624618, "deviation-center-line": 0.08207746311727626, "driven_lanedir_consec": 0.11923943865150877, "sim_compute_sim_state": 0.004191284594328507, "sim_compute_performance-ego": 0.00664054829141368, "sim_compute_robot_state-ego": 0.008786927098813265}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 1.1297534593888594, "sim_physics": 0.100492058030094, "survival_time": 4.149999999999993, "driven_lanedir": 0.4086431288592347, "sim_render-ego": 0.01162081166922328, "in-drivable-lane": 2.0999999999999956, "agent_compute-ego": 0.02693394580519343, "deviation-heading": 1.3490783429704527, "set_robot_commands": 0.009542338819388884, "deviation-center-line": 0.18061374091335075, "driven_lanedir_consec": 0.4086431288592347, "sim_compute_sim_state": 0.004717832588287721, "sim_compute_performance-ego": 0.007002787417676075, "sim_compute_robot_state-ego": 0.008690811065306147}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 2.158857000199323, "sim_physics": 0.09572557078467477, "survival_time": 8.999999999999993, "driven_lanedir": 1.4193283265977223, "sim_render-ego": 0.01066763136121962, "in-drivable-lane": 2.1499999999999924, "agent_compute-ego": 0.025754521952735053, "deviation-heading": 3.4772722656270423, "set_robot_commands": 0.008247294690873888, "deviation-center-line": 0.469380609026831, "driven_lanedir_consec": 1.4193283265977223, "sim_compute_sim_state": 0.0045533988210890025, "sim_compute_performance-ego": 0.006134418646494547, "sim_compute_robot_state-ego": 0.007897550529903836}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.21913706190354543, "sim_physics": 0.07631855744581956, "survival_time": 1.3000000000000005, "driven_lanedir": 0.13732703139902205, "sim_render-ego": 0.008686120693500225, "in-drivable-lane": 0, "agent_compute-ego": 0.02009568764613225, "deviation-heading": 0.7390595710504796, "set_robot_commands": 0.006475916275611291, "deviation-center-line": 0.06780723411796019, "driven_lanedir_consec": 0.13732703139902205, "sim_compute_sim_state": 0.017984931285564717, "sim_compute_performance-ego": 0.005044817924499512, "sim_compute_robot_state-ego": 0.006158847075242262}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.9053666680722884, "sim_physics": 0.1472934226731996, "survival_time": 3.699999999999995, "driven_lanedir": 0.3891331688028239, "sim_render-ego": 0.01463242479272791, "in-drivable-lane": 1.7499999999999971, "agent_compute-ego": 0.04308545267259752, "deviation-heading": 1.1407295395512222, "set_robot_commands": 0.013950051488103094, "deviation-center-line": 0.15947327091615762, "driven_lanedir_consec": 0.3891331688028239, "sim_compute_sim_state": 0.007172800399161674, "sim_compute_performance-ego": 0.008532894624246133, "sim_compute_robot_state-ego": 0.012043208689303012}}
set_robot_commands_max0.013950051488103094
set_robot_commands_mean0.009085289361320355
set_robot_commands_median0.008247294690873888
set_robot_commands_min0.006475916275611291
sim_compute_performance-ego_max0.008532894624246133
sim_compute_performance-ego_mean0.006671093380865989
sim_compute_performance-ego_median0.00664054829141368
sim_compute_performance-ego_min0.005044817924499512
sim_compute_robot_state-ego_max0.012043208689303012
sim_compute_robot_state-ego_mean0.008715468891713703
sim_compute_robot_state-ego_median0.008690811065306147
sim_compute_robot_state-ego_min0.006158847075242262
sim_compute_sim_state_max0.017984931285564717
sim_compute_sim_state_mean0.0077240495376863245
sim_compute_sim_state_median0.004717832588287721
sim_compute_sim_state_min0.004191284594328507
sim_physics_max0.1472934226731996
sim_physics_mean0.10238779977217284
sim_physics_median0.09572557078467477
sim_physics_min0.07631855744581956
sim_render-ego_max0.01463242479272791
sim_render-ego_mean0.011187110176001071
sim_render-ego_median0.01066763136121962
sim_render-ego_min0.008686120693500225
simulation-passed1
survival_time_max8.999999999999993
survival_time_mean3.8599999999999968
survival_time_min1.1500000000000004
No reset possible
248934255Manfred Diazchallenge-aido_LF-template-pytorchaido3-LF-sim-validationstep1-simulationsuccessnoip-172-31-43-40-104980:07:01
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.08056768880998155
survival_time_median1.800000000000001
deviation-center-line_median0.06132282461143106
in-drivable-lane_median0.6500000000000006


other stats
agent_compute-ego_max0.03466300199960024
agent_compute-ego_mean0.031052095615678826
agent_compute-ego_median0.030310008455725276
agent_compute-ego_min0.028665681680043537
deviation-center-line_max0.1417155604598797
deviation-center-line_mean0.06913123271983992
deviation-center-line_min0.03352157435235977
deviation-heading_max2.4704076098066623
deviation-heading_mean1.3335073649356657
deviation-heading_median0.7606365422353495
deviation-heading_min0.5575221135947229
driven_any_max0.7097580398011447
driven_any_mean0.37284293127634144
driven_any_median0.1934040750712179
driven_any_min0.09498093230670936
driven_lanedir_consec_max0.16011444334814007
driven_lanedir_consec_mean0.0949860996381728
driven_lanedir_consec_min0.026753060432728848
driven_lanedir_max0.16011444334814007
driven_lanedir_mean0.0949860996381728
driven_lanedir_median0.08056768880998155
driven_lanedir_min0.026753060432728848
in-drivable-lane_max3.4499999999999917
in-drivable-lane_mean1.7099999999999969
in-drivable-lane_min0.5000000000000003
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.7097580398011447, "sim_physics": 0.10074802531915553, "survival_time": 6.799999999999984, "driven_lanedir": 0.16011444334814007, "sim_render-ego": 0.012653605026357314, "in-drivable-lane": 3.4499999999999917, "agent_compute-ego": 0.030310008455725276, "deviation-heading": 2.4704076098066623, "set_robot_commands": 0.008861205157111673, "deviation-center-line": 0.1417155604598797, "driven_lanedir_consec": 0.16011444334814007, "sim_compute_sim_state": 0.004824357874253217, "sim_compute_performance-ego": 0.007398054880254409, "sim_compute_robot_state-ego": 0.009307223207810343}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.1879390379383864, "sim_physics": 0.13241224156485665, "survival_time": 1.800000000000001, "driven_lanedir": 0.08056768880998155, "sim_render-ego": 0.012453814347585045, "in-drivable-lane": 0.5500000000000005, "agent_compute-ego": 0.028665681680043537, "deviation-heading": 0.6953003291869926, "set_robot_commands": 0.01060526900821262, "deviation-center-line": 0.03352157435235977, "driven_lanedir_consec": 0.08056768880998155, "sim_compute_sim_state": 0.00511320432027181, "sim_compute_performance-ego": 0.007108767827351888, "sim_compute_robot_state-ego": 0.009402215480804443}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.09498093230670936, "sim_physics": 0.1189560713591399, "survival_time": 1.3500000000000003, "driven_lanedir": 0.026753060432728848, "sim_render-ego": 0.012623106991803204, "in-drivable-lane": 0.5000000000000003, "agent_compute-ego": 0.030250655280219182, "deviation-heading": 0.5575221135947229, "set_robot_commands": 0.009400420718722872, "deviation-center-line": 0.06132282461143106, "driven_lanedir_consec": 0.026753060432728848, "sim_compute_sim_state": 0.005497861791540075, "sim_compute_performance-ego": 0.007157087326049805, "sim_compute_robot_state-ego": 0.009745059189973052}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.1934040750712179, "sim_physics": 0.11266462943133186, "survival_time": 1.7000000000000008, "driven_lanedir": 0.06553472804895577, "sim_render-ego": 0.013204230981714584, "in-drivable-lane": 0.6500000000000006, "agent_compute-ego": 0.03137113066280589, "deviation-heading": 0.7606365422353495, "set_robot_commands": 0.010074790786294375, "deviation-center-line": 0.04374852105206673, "driven_lanedir_consec": 0.06553472804895577, "sim_compute_sim_state": 0.005605473237879136, "sim_compute_performance-ego": 0.007828698438756606, "sim_compute_robot_state-ego": 0.010075863669900334}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.678132571264249, "sim_physics": 0.11536436044532834, "survival_time": 6.549999999999985, "driven_lanedir": 0.14196057755105773, "sim_render-ego": 0.01195361960025234, "in-drivable-lane": 3.3999999999999924, "agent_compute-ego": 0.03466300199960024, "deviation-heading": 2.183670229854601, "set_robot_commands": 0.009219844832675147, "deviation-center-line": 0.06534768312346237, "driven_lanedir_consec": 0.14196057755105773, "sim_compute_sim_state": 0.004927744392220301, "sim_compute_performance-ego": 0.007473144822448264, "sim_compute_robot_state-ego": 0.009644863259701328}}
set_robot_commands_max0.01060526900821262
set_robot_commands_mean0.009632306100603338
set_robot_commands_median0.009400420718722872
set_robot_commands_min0.008861205157111673
sim_compute_performance-ego_max0.007828698438756606
sim_compute_performance-ego_mean0.007393150658972195
sim_compute_performance-ego_median0.007398054880254409
sim_compute_performance-ego_min0.007108767827351888
sim_compute_robot_state-ego_max0.010075863669900334
sim_compute_robot_state-ego_mean0.0096350449616379
sim_compute_robot_state-ego_median0.009644863259701328
sim_compute_robot_state-ego_min0.009307223207810343
sim_compute_sim_state_max0.005605473237879136
sim_compute_sim_state_mean0.005193728323232907
sim_compute_sim_state_median0.00511320432027181
sim_compute_sim_state_min0.004824357874253217
sim_physics_max0.13241224156485665
sim_physics_mean0.11602906562396248
sim_physics_median0.11536436044532834
sim_physics_min0.10074802531915553
sim_render-ego_max0.013204230981714584
sim_render-ego_mean0.012577675389542495
sim_render-ego_median0.012623106991803204
sim_render-ego_min0.01195361960025234
simulation-passed1
survival_time_max6.799999999999984
survival_time_mean3.6399999999999944
survival_time_min1.3500000000000003
No reset possible
248634288Gianmarco BernasconiBaseline-IL-logs-tensorflowaido3-LF-sim-validationstep1-simulationerrornoip-172-31-43-40-104980:15:00
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
248504306Frank (Chude) Qian 🇨🇦Frank test submissionaido3-LFVI-sim-testingstep1-simulationsuccessnoip-172-31-43-40-104980:18:25
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.4541194076078226
survival_time_median2.0500000000000007
deviation-center-line_median0.1184981700628623
in-drivable-lane_median0


other stats
agent_compute-ego_max0.03143821443830218
agent_compute-ego_mean0.02786781375720278
agent_compute-ego_median0.02843717011538419
agent_compute-ego_min0.021954786209833058
deviation-center-line_max0.27212935355303575
deviation-center-line_mean0.12800192803639482
deviation-center-line_min0.054904029848945685
deviation-heading_max0.9814018754699168
deviation-heading_mean0.39865622452675337
deviation-heading_median0.2921306476839709
deviation-heading_min0.13989224355569133
driven_any_max2.7696546371463566
driven_any_mean1.0880644329743805
driven_any_median1.0296612854712663
driven_any_min0.37149985167138017
driven_lanedir_consec_max1.9242206205397283
driven_lanedir_consec_mean0.7675641832273904
driven_lanedir_consec_min0.33418991107979124
driven_lanedir_max1.924267585170248
driven_lanedir_mean0.7820426634438573
driven_lanedir_median0.618479321964448
driven_lanedir_min0.33418991107979124
in-drivable-lane_max1.899999999999998
in-drivable-lane_mean0.4266666666666661
in-drivable-lane_min0
per-episodes
details{"udem1-0-0": {"driven_any": 1.5696547037668402, "sim_physics": 0.217267678955854, "survival_time": 2.9499999999999975, "driven_lanedir": 1.5638383114510817, "sim_render-ego": 0.01314499822713561, "in-drivable-lane": 0, "agent_compute-ego": 0.0296628475189209, "deviation-heading": 0.2528567413804248, "set_robot_commands": 0.010536113027798928, "deviation-center-line": 0.19417316650714284, "driven_lanedir_consec": 1.5636399177084477, "sim_compute_sim_state": 0.005192114134966317, "sim_compute_performance-ego": 0.007190029499894481, "sim_compute_robot_state-ego": 0.009040282944501456, "sim_compute_robot_state-npc0": 0.00910476910865913, "sim_compute_robot_state-npc1": 0.00880102383888374, "sim_compute_robot_state-npc2": 0.008758419651096151, "sim_compute_robot_state-npc3": 0.008722337625794491}, "udem1-1-0": {"driven_any": 0.46051146883057115, "sim_physics": 0.20499862324107776, "survival_time": 1.1000000000000003, "driven_lanedir": 0.43495908029179065, "sim_render-ego": 0.013018651442094282, "in-drivable-lane": 0, "agent_compute-ego": 0.02843717011538419, "deviation-heading": 0.3681646786246837, "set_robot_commands": 0.010177265514026987, "deviation-center-line": 0.07726482156623102, "driven_lanedir_consec": 0.43495908029179065, "sim_compute_sim_state": 0.004806171764026989, "sim_compute_performance-ego": 0.007081541148099032, "sim_compute_robot_state-ego": 0.009024771777066317, "sim_compute_robot_state-npc0": 0.008828488263216886, "sim_compute_robot_state-npc1": 0.008469754999334162, "sim_compute_robot_state-npc2": 0.008287516507235441, "sim_compute_robot_state-npc3": 0.008361057801680132}, "udem1-2-0": {"driven_any": 2.1696546375441614, "sim_physics": 0.24801306181316135, "survival_time": 3.949999999999994, "driven_lanedir": 1.7414407758715231, "sim_render-ego": 0.013353857813002188, "in-drivable-lane": 0.6999999999999975, "agent_compute-ego": 0.03051507925685448, "deviation-heading": 0.2785063205851342, "set_robot_commands": 0.011076290396195424, "deviation-center-line": 0.22563902167216496, "driven_lanedir_consec": 1.7413838620089876, "sim_compute_sim_state": 0.005476513995399958, "sim_compute_performance-ego": 0.008242097082017343, "sim_compute_robot_state-ego": 0.010278744033620328, "sim_compute_robot_state-npc0": 0.00954042205327674, "sim_compute_robot_state-npc1": 0.010072080394889734, "sim_compute_robot_state-npc2": 0.0094693401191808, "sim_compute_robot_state-npc3": 0.009823962102962446}, "udem1-3-0": {"driven_any": 2.7696546371463566, "sim_physics": 0.2150596151448259, "survival_time": 4.94999999999999, "driven_lanedir": 1.924267585170248, "sim_render-ego": 0.012946557517003531, "in-drivable-lane": 1.399999999999995, "agent_compute-ego": 0.0298099879062537, "deviation-heading": 0.2652784439875202, "set_robot_commands": 0.010335782561639344, "deviation-center-line": 0.27212935355303575, "driven_lanedir_consec": 1.9242206205397283, "sim_compute_sim_state": 0.00523907488042658, "sim_compute_performance-ego": 0.0076666865685973504, "sim_compute_robot_state-ego": 0.009579058849450314, "sim_compute_robot_state-npc0": 0.00914331879278626, "sim_compute_robot_state-npc1": 0.00886266159288811, "sim_compute_robot_state-npc2": 0.008666101128163965, "sim_compute_robot_state-npc3": 0.008745330752748432}, "udem1-4-0": {"driven_any": 0.4010834948810282, "sim_physics": 0.24405307769775392, "survival_time": 1.0000000000000002, "driven_lanedir": 0.38720104799654376, "sim_render-ego": 0.01519261598587036, "in-drivable-lane": 0, "agent_compute-ego": 0.028868448734283448, "deviation-heading": 0.2638670899443274, "set_robot_commands": 0.010673105716705322, "deviation-center-line": 0.060733287044270105, "driven_lanedir_consec": 0.38720104799654376, "sim_compute_sim_state": 0.005671179294586182, "sim_compute_performance-ego": 0.007608318328857421, "sim_compute_robot_state-ego": 0.010182416439056397, "sim_compute_robot_state-npc0": 0.009010839462280273, "sim_compute_robot_state-npc1": 0.009559321403503416, "sim_compute_robot_state-npc2": 0.010382580757141113, "sim_compute_robot_state-npc3": 0.009535491466522217}, "udem1-5-0": {"driven_any": 1.0596597849416796, "sim_physics": 0.22592346441178096, "survival_time": 2.1000000000000005, "driven_lanedir": 0.419433267065787, "sim_render-ego": 0.013413900420779274, "in-drivable-lane": 0.8500000000000006, "agent_compute-ego": 0.03143821443830218, "deviation-heading": 0.7002960066861377, "set_robot_commands": 0.0114945684160505, "deviation-center-line": 0.1184981700628623, "driven_lanedir_consec": 0.419433267065787, "sim_compute_sim_state": 0.0053388334455944245, "sim_compute_performance-ego": 0.008028705914815268, "sim_compute_robot_state-ego": 0.009674463953290666, "sim_compute_robot_state-npc0": 0.009202446256365095, "sim_compute_robot_state-npc1": 0.00964052336556571, "sim_compute_robot_state-npc2": 0.009337862332661947, "sim_compute_robot_state-npc3": 0.009333258583432152}, "udem1-6-0": {"driven_any": 0.8796785144013353, "sim_physics": 0.2333569725354513, "survival_time": 1.800000000000001, "driven_lanedir": 0.8770246919234508, "sim_render-ego": 0.011727637714809842, "in-drivable-lane": 0, "agent_compute-ego": 0.026141835583580863, "deviation-heading": 0.13989224355569133, "set_robot_commands": 0.009589857525295682, "deviation-center-line": 0.15455097642444732, "driven_lanedir_consec": 0.8770246919234508, "sim_compute_sim_state": 0.005198465453253852, "sim_compute_performance-ego": 0.00633043713039822, "sim_compute_robot_state-ego": 0.00868286689122518, "sim_compute_robot_state-npc0": 0.00821122858259413, "sim_compute_robot_state-npc1": 0.008040189743041992, "sim_compute_robot_state-npc2": 0.008180565304226346, "sim_compute_robot_state-npc3": 0.008459230264027914}, "udem1-7-0": {"driven_any": 1.0296612854712832, "sim_physics": 0.25930719259308604, "survival_time": 2.0500000000000007, "driven_lanedir": 0.8977146610328521, "sim_render-ego": 0.013211791108294232, "in-drivable-lane": 0, "agent_compute-ego": 0.02819621272203399, "deviation-heading": 0.7301693312178688, "set_robot_commands": 0.010175658435356326, "deviation-center-line": 0.12694367940927642, "driven_lanedir_consec": 0.8807130056354433, "sim_compute_sim_state": 0.005072564613528368, "sim_compute_performance-ego": 0.00697235363285716, "sim_compute_robot_state-ego": 0.00951783250017864, "sim_compute_robot_state-npc0": 0.008701481470247595, "sim_compute_robot_state-npc1": 0.00855763365582722, "sim_compute_robot_state-npc2": 0.008694544071104468, "sim_compute_robot_state-npc3": 0.008437964974380121}, "udem1-8-0": {"driven_any": 1.0296612854712663, "sim_physics": 0.16376610500056568, "survival_time": 2.0500000000000007, "driven_lanedir": 0.618479321964448, "sim_render-ego": 0.011131228470220798, "in-drivable-lane": 0.5000000000000004, "agent_compute-ego": 0.026604210458150725, "deviation-heading": 0.6847850662034057, "set_robot_commands": 0.008804565522728897, "deviation-center-line": 0.13317992938688095, "driven_lanedir_consec": 0.618479321964448, "sim_compute_sim_state": 0.004425246541093036, "sim_compute_performance-ego": 0.006260743955286537, "sim_compute_robot_state-ego": 0.00813965099613841, "sim_compute_robot_state-npc0": 0.007862108509715011, "sim_compute_robot_state-npc1": 0.007320520354480278, "sim_compute_robot_state-npc2": 0.007174666334943074, "sim_compute_robot_state-npc3": 0.007080880607046732}, "udem1-9-0": {"driven_any": 0.4605114688305781, "sim_physics": 0.21167229522358288, "survival_time": 1.1000000000000003, "driven_lanedir": 0.4443672320455454, "sim_render-ego": 0.010878389531915838, "in-drivable-lane": 0, "agent_compute-ego": 0.02583988146348433, "deviation-heading": 0.2921306476839709, "set_robot_commands": 0.008539774201133034, "deviation-center-line": 0.06393804025783281, "driven_lanedir_consec": 0.4443672320455454, "sim_compute_sim_state": 0.004302349957552823, "sim_compute_performance-ego": 0.0060473463752053, "sim_compute_robot_state-ego": 0.007345676422119141, "sim_compute_robot_state-npc0": 0.008016813885081898, "sim_compute_robot_state-npc1": 0.007335706190629439, "sim_compute_robot_state-npc2": 0.007582675326954235, "sim_compute_robot_state-npc3": 0.007823586463928223}, "udem1-10-0": {"driven_any": 0.37149985167138017, "sim_physics": 0.1836894436886436, "survival_time": 0.9500000000000004, "driven_lanedir": 0.3517991142132264, "sim_render-ego": 0.010341041966488489, "in-drivable-lane": 0, "agent_compute-ego": 0.026150301883095188, "deviation-heading": 0.31076793256366336, "set_robot_commands": 0.00868015540273566, "deviation-center-line": 0.054904029848945685, "driven_lanedir_consec": 0.3517991142132264, "sim_compute_sim_state": 0.004669013776277241, "sim_compute_performance-ego": 0.006658315658569336, "sim_compute_robot_state-ego": 0.008119143937763414, "sim_compute_robot_state-npc0": 0.007580832431190892, "sim_compute_robot_state-npc1": 0.007744789123535156, "sim_compute_robot_state-npc2": 0.008090307838038394, "sim_compute_robot_state-npc3": 0.007660727751882453}, "udem1-11-0": {"driven_any": 0.4307612897755775, "sim_physics": 0.13931629771278017, "survival_time": 1.0500000000000005, "driven_lanedir": 0.4261291824030562, "sim_render-ego": 0.00946369625273205, "in-drivable-lane": 0, "agent_compute-ego": 0.021954786209833058, "deviation-heading": 0.1541202345559414, "set_robot_commands": 0.007562149138677688, "deviation-center-line": 0.09941469755038562, "driven_lanedir_consec": 0.4261291824030562, "sim_compute_sim_state": 0.0036733150482177734, "sim_compute_performance-ego": 0.0053220703488304506, "sim_compute_robot_state-ego": 0.006765626725696382, "sim_compute_robot_state-npc0": 0.006813367207845052, "sim_compute_robot_state-npc1": 0.006676378704252697, "sim_compute_robot_state-npc2": 0.006370862325032552, "sim_compute_robot_state-npc3": 0.006348916462489537}, "udem1-12-0": {"driven_any": 1.4796547806284426, "sim_physics": 0.216578585760934, "survival_time": 2.799999999999998, "driven_lanedir": 0.33418991107979124, "sim_render-ego": 0.013357843671526228, "in-drivable-lane": 1.899999999999998, "agent_compute-ego": 0.028951338359287808, "deviation-heading": 0.193170346576385, "set_robot_commands": 0.0106216413634164, "deviation-center-line": 0.09687090955952632, "driven_lanedir_consec": 0.33418991107979124, "sim_compute_sim_state": 0.0049084510122026715, "sim_compute_performance-ego": 0.0076387907777513775, "sim_compute_robot_state-ego": 0.009619981050491331, "sim_compute_robot_state-npc0": 0.008952230215072632, "sim_compute_robot_state-npc1": 0.009030376161847795, "sim_compute_robot_state-npc2": 0.008127974612372262, "sim_compute_robot_state-npc3": 0.008556898151125227}, "udem1-13-0": {"driven_any": 0.9996632207705788, "sim_physics": 0.2131981432437897, "survival_time": 2.000000000000001, "driven_lanedir": 0.655803085926784, "sim_render-ego": 0.012486308813095093, "in-drivable-lane": 0.4500000000000002, "agent_compute-ego": 0.029113858938217163, "deviation-heading": 0.36443640886622947, "set_robot_commands": 0.0106983482837677, "deviation-center-line": 0.1026590609494026, "driven_lanedir_consec": 0.655803085926784, "sim_compute_sim_state": 0.004938793182373047, "sim_compute_performance-ego": 0.007447725534439087, "sim_compute_robot_state-ego": 0.009229671955108643, "sim_compute_robot_state-npc0": 0.0091363787651062, "sim_compute_robot_state-npc1": 0.009181034564971925, "sim_compute_robot_state-npc2": 0.009121423959732056, "sim_compute_robot_state-npc3": 0.008049935102462769}, "udem1-14-0": {"driven_any": 1.2096560704846229, "sim_physics": 0.21645738216156657, "survival_time": 2.3499999999999996, "driven_lanedir": 0.6539926832217301, "sim_render-ego": 0.011646316406574654, "in-drivable-lane": 0.5999999999999994, "agent_compute-ego": 0.02633303277035977, "deviation-heading": 0.9814018754699168, "set_robot_commands": 0.00901518984043852, "deviation-center-line": 0.1391297767535178, "driven_lanedir_consec": 0.4541194076078226, "sim_compute_sim_state": 0.004746863182554854, "sim_compute_performance-ego": 0.006870345866426509, "sim_compute_robot_state-ego": 0.008792588051329268, "sim_compute_robot_state-npc0": 0.008256283212215342, "sim_compute_robot_state-npc1": 0.00801360860784003, "sim_compute_robot_state-npc2": 0.00801485650082852, "sim_compute_robot_state-npc3": 0.007911316891934009}}
set_robot_commands_max0.0114945684160505
set_robot_commands_mean0.009865364356397762
set_robot_commands_median0.010177265514026987
set_robot_commands_min0.007562149138677688
sim_compute_performance-ego_max0.008242097082017343
sim_compute_performance-ego_mean0.007024367188136327
sim_compute_performance-ego_median0.007081541148099032
sim_compute_performance-ego_min0.0053220703488304506
sim_compute_robot_state-ego_max0.010278744033620328
sim_compute_robot_state-ego_mean0.00893285176846906
sim_compute_robot_state-ego_median0.009040282944501456
sim_compute_robot_state-ego_min0.006765626725696382
sim_compute_robot_state-npc0_max0.00954042205327674
sim_compute_robot_state-npc0_mean0.008557400547710209
sim_compute_robot_state-npc0_median0.008828488263216886
sim_compute_robot_state-npc0_min0.006813367207845052
sim_compute_robot_state-npc1_max0.010072080394889734
sim_compute_robot_state-npc1_mean0.008487040180099428
sim_compute_robot_state-npc1_median0.00855763365582722
sim_compute_robot_state-npc1_min0.006676378704252697
sim_compute_robot_state-npc2_max0.010382580757141113
sim_compute_robot_state-npc2_mean0.008417313117914087
sim_compute_robot_state-npc2_median0.008287516507235441
sim_compute_robot_state-npc2_min0.006370862325032552
sim_compute_robot_state-npc3_max0.009823962102962446
sim_compute_robot_state-npc3_mean0.008323393000161125
sim_compute_robot_state-npc3_median0.008437964974380121
sim_compute_robot_state-npc3_min0.006348916462489537
sim_compute_sim_state_max0.005671179294586182
sim_compute_sim_state_mean0.004910596685470275
sim_compute_sim_state_median0.004938793182373047
sim_compute_sim_state_min0.0036733150482177734
sim_physics_max0.25930719259308604
sim_physics_mean0.2128438626123236
sim_physics_median0.21645738216156657
sim_physics_min0.13931629771278017
sim_render-ego_max0.01519261598587036
sim_render-ego_mean0.012354322356102832
sim_render-ego_median0.012946557517003531
sim_render-ego_min0.00946369625273205
simulation-passed1
survival_time_max4.94999999999999
survival_time_mean2.1466666666666656
survival_time_min0.9500000000000004
No reset possible
248354334Soroush Saryazdi 🇨🇦challenge-aido_LF-template-randomaido3-LFVI-sim-testingstep1-simulationsuccessnoip-172-31-43-40-104980:24:25
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.6537162971975778
survival_time_median2.5999999999999988
deviation-center-line_median0.14884708769804722
in-drivable-lane_median0.549999999999998


other stats
agent_compute-ego_max0.025850569948236993
agent_compute-ego_mean0.024715174340246
agent_compute-ego_median0.02485663195451101
agent_compute-ego_min0.021362684950043884
deviation-center-line_max0.5141120095458045
deviation-center-line_mean0.17690002270053232
deviation-center-line_min0.06216029860587694
deviation-heading_max1.992943725008188
deviation-heading_mean0.6929778233956081
deviation-heading_median0.4928440244444201
deviation-heading_min0.24234723632033853
driven_any_max3.534324402241835
driven_any_mean1.3470385992234173
driven_any_median1.001633421251193
driven_any_min0.3412551960104244
driven_lanedir_consec_max2.4943080534418787
driven_lanedir_consec_mean0.8813910855277101
driven_lanedir_consec_min0.3199180625488627
driven_lanedir_max2.4943080534418787
driven_lanedir_mean0.8962528018383177
driven_lanedir_median0.7206363324756618
driven_lanedir_min0.3199180625488627
in-drivable-lane_max3.549999999999991
in-drivable-lane_mean0.839999999999998
in-drivable-lane_min0
per-episodes
details{"udem1-0-0": {"driven_any": 1.5811782993597916, "sim_physics": 0.20232305345655996, "survival_time": 3.949999999999994, "driven_lanedir": 0.9217908232230632, "sim_render-ego": 0.011781668361229233, "in-drivable-lane": 1.3499999999999952, "agent_compute-ego": 0.021362684950043884, "deviation-heading": 0.5694288623982039, "set_robot_commands": 0.008406518380853194, "deviation-center-line": 0.2348196785686717, "driven_lanedir_consec": 0.9213701538661684, "sim_compute_sim_state": 0.006908247742471816, "sim_compute_performance-ego": 0.006244019617008258, "sim_compute_robot_state-ego": 0.008858225013636336, "sim_compute_robot_state-npc0": 0.007294084452375581, "sim_compute_robot_state-npc1": 0.007246325287637832, "sim_compute_robot_state-npc2": 0.007790746568124505, "sim_compute_robot_state-npc3": 0.00721693944327439}, "udem1-1-0": {"driven_any": 0.47798123795353264, "sim_physics": 0.2184592740876334, "survival_time": 1.4000000000000006, "driven_lanedir": 0.4500753586658095, "sim_render-ego": 0.013484665325709752, "in-drivable-lane": 0, "agent_compute-ego": 0.024483706269945418, "deviation-heading": 0.46504862964833654, "set_robot_commands": 0.01076873711177281, "deviation-center-line": 0.09791133281082, "driven_lanedir_consec": 0.4500753586658095, "sim_compute_sim_state": 0.0055176786014011926, "sim_compute_performance-ego": 0.007785252162388393, "sim_compute_robot_state-ego": 0.010342504296983992, "sim_compute_robot_state-npc0": 0.008704994406018938, "sim_compute_robot_state-npc1": 0.00897519929068429, "sim_compute_robot_state-npc2": 0.00873875617980957, "sim_compute_robot_state-npc3": 0.00950625113078526}, "udem1-2-0": {"driven_any": 2.1864003300000703, "sim_physics": 0.23345372977766016, "survival_time": 5.14999999999999, "driven_lanedir": 1.8262319453635671, "sim_render-ego": 0.014033650888980013, "in-drivable-lane": 0.549999999999998, "agent_compute-ego": 0.0256478763321071, "deviation-heading": 0.8115272458314912, "set_robot_commands": 0.010595881823197153, "deviation-center-line": 0.19273160416094068, "driven_lanedir_consec": 1.8262319453635671, "sim_compute_sim_state": 0.004995424770614476, "sim_compute_performance-ego": 0.00751305089413541, "sim_compute_robot_state-ego": 0.010439428310949826, "sim_compute_robot_state-npc0": 0.009489390456560746, "sim_compute_robot_state-npc1": 0.00938954631101738, "sim_compute_robot_state-npc2": 0.008499751970605943, "sim_compute_robot_state-npc3": 0.00888246008493368}, "udem1-3-0": {"driven_any": 0.9410513078477728, "sim_physics": 0.25338410834471387, "survival_time": 2.3999999999999995, "driven_lanedir": 0.493517638549009, "sim_render-ego": 0.014231552680333456, "in-drivable-lane": 0.8999999999999988, "agent_compute-ego": 0.02485663195451101, "deviation-heading": 0.35154027506548413, "set_robot_commands": 0.011003106832504272, "deviation-center-line": 0.08975118546699376, "driven_lanedir_consec": 0.493517638549009, "sim_compute_sim_state": 0.005404815077781677, "sim_compute_performance-ego": 0.007859677076339722, "sim_compute_robot_state-ego": 0.010373935103416445, "sim_compute_robot_state-npc0": 0.009568899869918823, "sim_compute_robot_state-npc1": 0.00921742618083954, "sim_compute_robot_state-npc2": 0.008734414974848429, "sim_compute_robot_state-npc3": 0.008971219261487326}, "udem1-4-0": {"driven_any": 0.34749292376309954, "sim_physics": 0.2573116847446987, "survival_time": 1.0500000000000005, "driven_lanedir": 0.3325249708009643, "sim_render-ego": 0.014280058088756743, "in-drivable-lane": 0, "agent_compute-ego": 0.02493322463262649, "deviation-heading": 0.30044835691402494, "set_robot_commands": 0.011203096026466006, "deviation-center-line": 0.06216029860587694, "driven_lanedir_consec": 0.3325249708009643, "sim_compute_sim_state": 0.00566856066385905, "sim_compute_performance-ego": 0.008753288359869094, "sim_compute_robot_state-ego": 0.011080265045166016, "sim_compute_robot_state-npc0": 0.009868826184953964, "sim_compute_robot_state-npc1": 0.009319918496268136, "sim_compute_robot_state-npc2": 0.00946413903009324, "sim_compute_robot_state-npc3": 0.00971070925394694}, "udem1-5-0": {"driven_any": 3.534324402241835, "sim_physics": 0.2383219453389059, "survival_time": 8.349999999999984, "driven_lanedir": 2.4943080534418787, "sim_render-ego": 0.013371476156269003, "in-drivable-lane": 2, "agent_compute-ego": 0.024635756087160395, "deviation-heading": 1.992943725008188, "set_robot_commands": 0.01036327327796799, "deviation-center-line": 0.5141120095458045, "driven_lanedir_consec": 2.4943080534418787, "sim_compute_sim_state": 0.005352301511935845, "sim_compute_performance-ego": 0.007945240614656917, "sim_compute_robot_state-ego": 0.010306675277070371, "sim_compute_robot_state-npc0": 0.009374879791351137, "sim_compute_robot_state-npc1": 0.009380927342854574, "sim_compute_robot_state-npc2": 0.009184413327428396, "sim_compute_robot_state-npc3": 0.009158439978867948}, "udem1-6-0": {"driven_any": 2.1760405509894407, "sim_physics": 0.2781348390486634, "survival_time": 5.14999999999999, "driven_lanedir": 1.9748003483573369, "sim_render-ego": 0.01410419501147224, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.02538844451163579, "deviation-heading": 0.7877172510198454, "set_robot_commands": 0.011501513638542694, "deviation-center-line": 0.30661477465861414, "driven_lanedir_consec": 1.9748003483573369, "sim_compute_sim_state": 0.005664089351024443, "sim_compute_performance-ego": 0.007799507344810708, "sim_compute_robot_state-ego": 0.01009642036215773, "sim_compute_robot_state-npc0": 0.009405321287877352, "sim_compute_robot_state-npc1": 0.009647746687953911, "sim_compute_robot_state-npc2": 0.009138186001083223, "sim_compute_robot_state-npc3": 0.009218359456478972}, "udem1-7-0": {"driven_any": 1.0050236108346402, "sim_physics": 0.3100187063217163, "survival_time": 2.499999999999999, "driven_lanedir": 0.8701796426684152, "sim_render-ego": 0.01459421157836914, "in-drivable-lane": 0, "agent_compute-ego": 0.025317907333374023, "deviation-heading": 0.9227446658975218, "set_robot_commands": 0.01161673069000244, "deviation-center-line": 0.14884708769804722, "driven_lanedir_consec": 0.8546116490381102, "sim_compute_sim_state": 0.005278639793395996, "sim_compute_performance-ego": 0.008444986343383788, "sim_compute_robot_state-ego": 0.010142617225646971, "sim_compute_robot_state-npc0": 0.010232491493225098, "sim_compute_robot_state-npc1": 0.009555644989013672, "sim_compute_robot_state-npc2": 0.00989034652709961, "sim_compute_robot_state-npc3": 0.009257659912109376}, "udem1-8-0": {"driven_any": 0.997041660301958, "sim_physics": 0.1922433055364169, "survival_time": 2.5999999999999988, "driven_lanedir": 0.8041152574540316, "sim_render-ego": 0.01285878511575552, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.02432416035578801, "deviation-heading": 0.6817155913229234, "set_robot_commands": 0.010113725295433631, "deviation-center-line": 0.1898768080806288, "driven_lanedir_consec": 0.8041152574540316, "sim_compute_sim_state": 0.005082566004533034, "sim_compute_performance-ego": 0.007258837039654072, "sim_compute_robot_state-ego": 0.009913540803469144, "sim_compute_robot_state-npc0": 0.00900627558047955, "sim_compute_robot_state-npc1": 0.008539053109975962, "sim_compute_robot_state-npc2": 0.00823589930167565, "sim_compute_robot_state-npc3": 0.008451567246363713}, "udem1-9-0": {"driven_any": 0.43749987104723, "sim_physics": 0.2759485886647151, "survival_time": 1.3000000000000005, "driven_lanedir": 0.4177527119659043, "sim_render-ego": 0.013837851010836087, "in-drivable-lane": 0, "agent_compute-ego": 0.024866250845102165, "deviation-heading": 0.3796137446285097, "set_robot_commands": 0.009135851493248573, "deviation-center-line": 0.08207813392324871, "driven_lanedir_consec": 0.4177527119659043, "sim_compute_sim_state": 0.005301658923809345, "sim_compute_performance-ego": 0.008071596805865947, "sim_compute_robot_state-ego": 0.010602327493520884, "sim_compute_robot_state-npc0": 0.008716876690204326, "sim_compute_robot_state-npc1": 0.009156758968646707, "sim_compute_robot_state-npc2": 0.009642885281489443, "sim_compute_robot_state-npc3": 0.009127974510192873}, "udem1-10-0": {"driven_any": 0.3412551960104244, "sim_physics": 0.242038141597401, "survival_time": 1.1000000000000003, "driven_lanedir": 0.3199180625488627, "sim_render-ego": 0.013857689770785246, "in-drivable-lane": 0, "agent_compute-ego": 0.025509335777976296, "deviation-heading": 0.3796986873122644, "set_robot_commands": 0.01157058369029652, "deviation-center-line": 0.06276575300040536, "driven_lanedir_consec": 0.3199180625488627, "sim_compute_sim_state": 0.005559639497236772, "sim_compute_performance-ego": 0.00832501324740323, "sim_compute_robot_state-ego": 0.01049682227048007, "sim_compute_robot_state-npc0": 0.009602557529102673, "sim_compute_robot_state-npc1": 0.009423472664572975, "sim_compute_robot_state-npc2": 0.009153474460948595, "sim_compute_robot_state-npc3": 0.00924307649785822}, "udem1-11-0": {"driven_any": 0.9951985237696338, "sim_physics": 0.1970945167541504, "survival_time": 2.499999999999999, "driven_lanedir": 0.7206363324756618, "sim_render-ego": 0.013355722427368164, "in-drivable-lane": 0.549999999999998, "agent_compute-ego": 0.024815678596496586, "deviation-heading": 0.30074636349085415, "set_robot_commands": 0.01040947914123535, "deviation-center-line": 0.16567670185555416, "driven_lanedir_consec": 0.718507549745529, "sim_compute_sim_state": 0.005792937278747559, "sim_compute_performance-ego": 0.007359013557434082, "sim_compute_robot_state-ego": 0.009937396049499512, "sim_compute_robot_state-npc0": 0.008900136947631835, "sim_compute_robot_state-npc1": 0.009350247383117676, "sim_compute_robot_state-npc2": 0.009259591102600098, "sim_compute_robot_state-npc3": 0.00851165771484375}, "udem1-12-0": {"driven_any": 1.9646878702534365, "sim_physics": 0.22632638951565356, "survival_time": 4.699999999999991, "driven_lanedir": 0.35162821871931205, "sim_render-ego": 0.013332823489574677, "in-drivable-lane": 3.549999999999991, "agent_compute-ego": 0.025850569948236993, "deviation-heading": 0.24234723632033853, "set_robot_commands": 0.011678665242296583, "deviation-center-line": 0.13126670634798973, "driven_lanedir_consec": 0.35162821871931205, "sim_compute_sim_state": 0.0053069566158538166, "sim_compute_performance-ego": 0.007612327311901336, "sim_compute_robot_state-ego": 0.010607280629746457, "sim_compute_robot_state-npc0": 0.009084478337714011, "sim_compute_robot_state-npc1": 0.009102902513869266, "sim_compute_robot_state-npc2": 0.008945845543070043, "sim_compute_robot_state-npc3": 0.00904383304271292}, "udem1-13-0": {"driven_any": 1.001633421251193, "sim_physics": 0.19767687412408683, "survival_time": 2.5999999999999988, "driven_lanedir": 0.6537162971975778, "sim_render-ego": 0.0127677871630742, "in-drivable-lane": 0.5999999999999979, "agent_compute-ego": 0.024521405880267803, "deviation-heading": 0.4928440244444201, "set_robot_commands": 0.01015008412874662, "deviation-center-line": 0.13361893417935136, "driven_lanedir_consec": 0.6537162971975778, "sim_compute_sim_state": 0.00529252107326801, "sim_compute_performance-ego": 0.007728425356057974, "sim_compute_robot_state-ego": 0.010223457446465124, "sim_compute_robot_state-npc0": 0.008691471356611986, "sim_compute_robot_state-npc1": 0.008782075001643254, "sim_compute_robot_state-npc2": 0.008987225019014798, "sim_compute_robot_state-npc3": 0.008711402232830342}, "udem1-14-0": {"driven_any": 2.218769782727205, "sim_physics": 0.2002817676180885, "survival_time": 5.249999999999989, "driven_lanedir": 0.8125963661433687, "sim_render-ego": 0.01231579099382673, "in-drivable-lane": 2.499999999999993, "agent_compute-ego": 0.02421398162841797, "deviation-heading": 1.7163026916317137, "set_robot_commands": 0.012319051651727586, "deviation-center-line": 0.2412693316050378, "driven_lanedir_consec": 0.6077880672015876, "sim_compute_sim_state": 0.004893273398989723, "sim_compute_performance-ego": 0.007063227608090355, "sim_compute_robot_state-ego": 0.00924428076971145, "sim_compute_robot_state-npc0": 0.009035335268293108, "sim_compute_robot_state-npc1": 0.008884198325020926, "sim_compute_robot_state-npc2": 0.009247818447294689, "sim_compute_robot_state-npc3": 0.008292000634329659}}
set_robot_commands_max0.012319051651727586
set_robot_commands_mean0.01072241989495276
set_robot_commands_median0.01076873711177281
set_robot_commands_min0.008406518380853194
sim_compute_performance-ego_max0.008753288359869094
sim_compute_performance-ego_mean0.007717564222599952
sim_compute_performance-ego_median0.007785252162388393
sim_compute_performance-ego_min0.006244019617008258
sim_compute_robot_state-ego_max0.011080265045166016
sim_compute_robot_state-ego_mean0.010177678406528024
sim_compute_robot_state-ego_median0.010306675277070371
sim_compute_robot_state-ego_min0.008858225013636336
sim_compute_robot_state-npc0_max0.010232491493225098
sim_compute_robot_state-npc0_mean0.009131734643487942
sim_compute_robot_state-npc0_median0.009084478337714011
sim_compute_robot_state-npc0_min0.007294084452375581
sim_compute_robot_state-npc1_max0.009647746687953911
sim_compute_robot_state-npc1_mean0.009064762836874409
sim_compute_robot_state-npc1_median0.00921742618083954
sim_compute_robot_state-npc1_min0.007246325287637832
sim_compute_robot_state-npc2_max0.00989034652709961
sim_compute_robot_state-npc2_mean0.00899423291567908
sim_compute_robot_state-npc2_median0.009138186001083223
sim_compute_robot_state-npc2_min0.007790746568124505
sim_compute_robot_state-npc3_max0.00971070925394694
sim_compute_robot_state-npc3_mean0.00888690336006769
sim_compute_robot_state-npc3_median0.00904383304271292
sim_compute_robot_state-npc3_min0.00721693944327439
sim_compute_sim_state_max0.006908247742471816
sim_compute_sim_state_mean0.005467954020328183
sim_compute_sim_state_median0.005352301511935845
sim_compute_sim_state_min0.004893273398989723
sim_physics_max0.3100187063217163
sim_physics_mean0.23486779499540428
sim_physics_median0.23345372977766016
sim_physics_min0.1922433055364169
sim_render-ego_max0.01459421157836914
sim_render-ego_mean0.013480528537489348
sim_render-ego_median0.013484665325709752
sim_render-ego_min0.011781668361229233
simulation-passed1
survival_time_max8.349999999999984
survival_time_mean3.3333333333333286
survival_time_min1.0500000000000005
No reset possible
248274346Frank (Chude) Qian 🇨🇦Frank raw run 1aido3-LF-sim-testingstep1-simulationerrornoip-172-31-43-40-104980:01:47
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
248134325Gianmarco Bernasconibaseline-IL-sim-tensorflowaido3-LF-sim-validationstep1-simulationerrornoip-172-31-43-40-104980:04:06
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
248004338Soroush Saryazdi 🇨🇦challenge-aido_LF-baseline-duckietownaido3-LF-sim-testingstep1-simulationerrornoip-172-31-43-40-104980:26:37
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
247954334Soroush Saryazdi 🇨🇦challenge-aido_LF-template-randomaido3-LFVI-sim-testingstep1-simulationsuccessnoip-172-31-43-40-104980:15:53
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.6537162971975778
survival_time_median2.5999999999999988
deviation-center-line_median0.14884708769804722
in-drivable-lane_median0.549999999999998


other stats
agent_compute-ego_max0.02390708500825906
agent_compute-ego_mean0.01694658444933337
agent_compute-ego_median0.014590144157409668
agent_compute-ego_min0.013117551803588867
deviation-center-line_max0.5141120095458045
deviation-center-line_mean0.17690002270053232
deviation-center-line_min0.06216029860587694
deviation-heading_max1.992943725008188
deviation-heading_mean0.6929778233956081
deviation-heading_median0.4928440244444201
deviation-heading_min0.24234723632033853
driven_any_max3.534324402241835
driven_any_mean1.3470385992234173
driven_any_median1.001633421251193
driven_any_min0.3412551960104244
driven_lanedir_consec_max2.4943080534418787
driven_lanedir_consec_mean0.8813910855277101
driven_lanedir_consec_min0.3199180625488627
driven_lanedir_max2.4943080534418787
driven_lanedir_mean0.8962528018383177
driven_lanedir_median0.7206363324756618
driven_lanedir_min0.3199180625488627
in-drivable-lane_max3.549999999999991
in-drivable-lane_mean0.839999999999998
in-drivable-lane_min0
per-episodes
details{"udem1-0-0": {"driven_any": 1.5811782993597916, "sim_physics": 0.1853358987011487, "survival_time": 3.949999999999994, "driven_lanedir": 0.9217908232230632, "sim_render-ego": 0.012926835048047805, "in-drivable-lane": 1.3499999999999952, "agent_compute-ego": 0.02390708500825906, "deviation-heading": 0.5694288623982039, "set_robot_commands": 0.008506011359299286, "deviation-center-line": 0.2348196785686717, "driven_lanedir_consec": 0.9213701538661684, "sim_compute_sim_state": 0.005276583418061462, "sim_compute_performance-ego": 0.006853547277329843, "sim_compute_robot_state-ego": 0.009203808217109003, "sim_compute_robot_state-npc0": 0.008433013022700442, "sim_compute_robot_state-npc1": 0.008142908917197699, "sim_compute_robot_state-npc2": 0.008507061608230011, "sim_compute_robot_state-npc3": 0.0081188769280156}, "udem1-1-0": {"driven_any": 0.47798123795353264, "sim_physics": 0.19696012565067836, "survival_time": 1.4000000000000006, "driven_lanedir": 0.4500753586658095, "sim_render-ego": 0.01246410608291626, "in-drivable-lane": 0, "agent_compute-ego": 0.023424557277134488, "deviation-heading": 0.46504862964833654, "set_robot_commands": 0.008249657494681222, "deviation-center-line": 0.09791133281082, "driven_lanedir_consec": 0.4500753586658095, "sim_compute_sim_state": 0.004769044263022286, "sim_compute_performance-ego": 0.006553862776075091, "sim_compute_robot_state-ego": 0.009195395878383092, "sim_compute_robot_state-npc0": 0.008258998394012451, "sim_compute_robot_state-npc1": 0.008003175258636475, "sim_compute_robot_state-npc2": 0.007983922958374023, "sim_compute_robot_state-npc3": 0.00787114245550973}, "udem1-2-0": {"driven_any": 2.1864003300000703, "sim_physics": 0.15888693031755466, "survival_time": 5.14999999999999, "driven_lanedir": 1.8262319453635671, "sim_render-ego": 0.011636273374835264, "in-drivable-lane": 0.549999999999998, "agent_compute-ego": 0.021465023744453506, "deviation-heading": 0.8115272458314912, "set_robot_commands": 0.0076593820331166095, "deviation-center-line": 0.19273160416094068, "driven_lanedir_consec": 1.8262319453635671, "sim_compute_sim_state": 0.004523744860899101, "sim_compute_performance-ego": 0.006269091541327319, "sim_compute_robot_state-ego": 0.008263314811928758, "sim_compute_robot_state-npc0": 0.007605152222716693, "sim_compute_robot_state-npc1": 0.007456749388315145, "sim_compute_robot_state-npc2": 0.007237707526938429, "sim_compute_robot_state-npc3": 0.007074103772061542}, "udem1-3-0": {"driven_any": 0.9410513078477728, "sim_physics": 0.16951794922351837, "survival_time": 2.3999999999999995, "driven_lanedir": 0.493517638549009, "sim_render-ego": 0.011662468314170836, "in-drivable-lane": 0.8999999999999988, "agent_compute-ego": 0.021905750036239624, "deviation-heading": 0.35154027506548413, "set_robot_commands": 0.00810512900352478, "deviation-center-line": 0.08975118546699376, "driven_lanedir_consec": 0.493517638549009, "sim_compute_sim_state": 0.004762013753255208, "sim_compute_performance-ego": 0.006520971655845642, "sim_compute_robot_state-ego": 0.008533204595247904, "sim_compute_robot_state-npc0": 0.008018682400385538, "sim_compute_robot_state-npc1": 0.00773833692073822, "sim_compute_robot_state-npc2": 0.007864981889724731, "sim_compute_robot_state-npc3": 0.007320051391919454}, "udem1-4-0": {"driven_any": 0.34749292376309954, "sim_physics": 0.1848512377057757, "survival_time": 1.0500000000000005, "driven_lanedir": 0.3325249708009643, "sim_render-ego": 0.01099380992707752, "in-drivable-lane": 0, "agent_compute-ego": 0.021774360111781528, "deviation-heading": 0.30044835691402494, "set_robot_commands": 0.008153529394240607, "deviation-center-line": 0.06216029860587694, "driven_lanedir_consec": 0.3325249708009643, "sim_compute_sim_state": 0.004491510845365978, "sim_compute_performance-ego": 0.006090504782540458, "sim_compute_robot_state-ego": 0.008773508526030042, "sim_compute_robot_state-npc0": 0.008286146890549432, "sim_compute_robot_state-npc1": 0.008478948048182897, "sim_compute_robot_state-npc2": 0.008277859006609236, "sim_compute_robot_state-npc3": 0.008037975856236048}, "udem1-5-0": {"driven_any": 3.534324402241835, "sim_physics": 0.10935937287564763, "survival_time": 8.349999999999984, "driven_lanedir": 2.4943080534418787, "sim_render-ego": 0.008153194438911484, "in-drivable-lane": 2, "agent_compute-ego": 0.016091807873663076, "deviation-heading": 1.992943725008188, "set_robot_commands": 0.005505394792842294, "deviation-center-line": 0.5141120095458045, "driven_lanedir_consec": 2.4943080534418787, "sim_compute_sim_state": 0.0033196817615075026, "sim_compute_performance-ego": 0.004385933904590721, "sim_compute_robot_state-ego": 0.0059407556842187206, "sim_compute_robot_state-npc0": 0.0055357053608237625, "sim_compute_robot_state-npc1": 0.00536686360479115, "sim_compute_robot_state-npc2": 0.00518190218303018, "sim_compute_robot_state-npc3": 0.005215894676254181}, "udem1-6-0": {"driven_any": 2.1760405509894407, "sim_physics": 0.0994493544680401, "survival_time": 5.14999999999999, "driven_lanedir": 1.9748003483573369, "sim_render-ego": 0.00697329900797131, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.01349326013361366, "deviation-heading": 0.7877172510198454, "set_robot_commands": 0.0043975820819151055, "deviation-center-line": 0.30661477465861414, "driven_lanedir_consec": 1.9748003483573369, "sim_compute_sim_state": 0.0026694154276431185, "sim_compute_performance-ego": 0.003636795340232479, "sim_compute_robot_state-ego": 0.004778558768114997, "sim_compute_robot_state-npc0": 0.004409165058321166, "sim_compute_robot_state-npc1": 0.0043001614728020235, "sim_compute_robot_state-npc2": 0.004276891356533014, "sim_compute_robot_state-npc3": 0.00426314872445412}, "udem1-7-0": {"driven_any": 1.0050236108346402, "sim_physics": 0.10486354351043702, "survival_time": 2.499999999999999, "driven_lanedir": 0.8701796426684152, "sim_render-ego": 0.007010407447814941, "in-drivable-lane": 0, "agent_compute-ego": 0.013803191184997558, "deviation-heading": 0.9227446658975218, "set_robot_commands": 0.004348073005676269, "deviation-center-line": 0.14884708769804722, "driven_lanedir_consec": 0.8546116490381102, "sim_compute_sim_state": 0.00272371768951416, "sim_compute_performance-ego": 0.003644204139709473, "sim_compute_robot_state-ego": 0.004879179000854492, "sim_compute_robot_state-npc0": 0.004495425224304199, "sim_compute_robot_state-npc1": 0.004268374443054199, "sim_compute_robot_state-npc2": 0.004253778457641601, "sim_compute_robot_state-npc3": 0.004254522323608398}, "udem1-8-0": {"driven_any": 0.997041660301958, "sim_physics": 0.09037903180489174, "survival_time": 2.5999999999999988, "driven_lanedir": 0.8041152574540316, "sim_render-ego": 0.006968594514406645, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.013898331385392409, "deviation-heading": 0.6817155913229234, "set_robot_commands": 0.004471540451049805, "deviation-center-line": 0.1898768080806288, "driven_lanedir_consec": 0.8041152574540316, "sim_compute_sim_state": 0.002725656215961163, "sim_compute_performance-ego": 0.003643590670365554, "sim_compute_robot_state-ego": 0.004916007702167218, "sim_compute_robot_state-npc0": 0.004593849182128906, "sim_compute_robot_state-npc1": 0.00429309789951031, "sim_compute_robot_state-npc2": 0.004269618254441481, "sim_compute_robot_state-npc3": 0.004275826307443472}, "udem1-9-0": {"driven_any": 0.43749987104723, "sim_physics": 0.11209022998809814, "survival_time": 1.3000000000000005, "driven_lanedir": 0.4177527119659043, "sim_render-ego": 0.007335873750539927, "in-drivable-lane": 0, "agent_compute-ego": 0.014590144157409668, "deviation-heading": 0.3796137446285097, "set_robot_commands": 0.004388873393719013, "deviation-center-line": 0.08207813392324871, "driven_lanedir_consec": 0.4177527119659043, "sim_compute_sim_state": 0.002733514859126165, "sim_compute_performance-ego": 0.0037086009979248047, "sim_compute_robot_state-ego": 0.004944379513080304, "sim_compute_robot_state-npc0": 0.004601084269010103, "sim_compute_robot_state-npc1": 0.004322226230914776, "sim_compute_robot_state-npc2": 0.004304427366990309, "sim_compute_robot_state-npc3": 0.004290461540222168}, "udem1-10-0": {"driven_any": 0.3412551960104244, "sim_physics": 0.0987938100641424, "survival_time": 1.1000000000000003, "driven_lanedir": 0.3199180625488627, "sim_render-ego": 0.007247740572149103, "in-drivable-lane": 0, "agent_compute-ego": 0.014034401286732069, "deviation-heading": 0.3796986873122644, "set_robot_commands": 0.004540313373912464, "deviation-center-line": 0.06276575300040536, "driven_lanedir_consec": 0.3199180625488627, "sim_compute_sim_state": 0.0027509711005470967, "sim_compute_performance-ego": 0.003692453557794745, "sim_compute_robot_state-ego": 0.004959604956886985, "sim_compute_robot_state-npc0": 0.00466399843042547, "sim_compute_robot_state-npc1": 0.0043399442325938835, "sim_compute_robot_state-npc2": 0.004314064979553223, "sim_compute_robot_state-npc3": 0.004323666745966131}, "udem1-11-0": {"driven_any": 0.9951985237696338, "sim_physics": 0.08971168041229248, "survival_time": 2.499999999999999, "driven_lanedir": 0.7206363324756618, "sim_render-ego": 0.006930956840515137, "in-drivable-lane": 0.549999999999998, "agent_compute-ego": 0.013375105857849122, "deviation-heading": 0.30074636349085415, "set_robot_commands": 0.00437645435333252, "deviation-center-line": 0.16567670185555416, "driven_lanedir_consec": 0.718507549745529, "sim_compute_sim_state": 0.002718653678894043, "sim_compute_performance-ego": 0.003642034530639648, "sim_compute_robot_state-ego": 0.004860315322875976, "sim_compute_robot_state-npc0": 0.004490747451782227, "sim_compute_robot_state-npc1": 0.004283428192138672, "sim_compute_robot_state-npc2": 0.004260835647583008, "sim_compute_robot_state-npc3": 0.004268383979797364}, "udem1-12-0": {"driven_any": 1.9646878702534365, "sim_physics": 0.09298893999546132, "survival_time": 4.699999999999991, "driven_lanedir": 0.35162821871931205, "sim_render-ego": 0.006878325279722822, "in-drivable-lane": 3.549999999999991, "agent_compute-ego": 0.015564507626472636, "deviation-heading": 0.24234723632033853, "set_robot_commands": 0.004330919144001413, "deviation-center-line": 0.13126670634798973, "driven_lanedir_consec": 0.35162821871931205, "sim_compute_sim_state": 0.002697023939579091, "sim_compute_performance-ego": 0.003632684971423859, "sim_compute_robot_state-ego": 0.004862189292907715, "sim_compute_robot_state-npc0": 0.00445886622083948, "sim_compute_robot_state-npc1": 0.004250528964590519, "sim_compute_robot_state-npc2": 0.004235011466006015, "sim_compute_robot_state-npc3": 0.004230808704457385}, "udem1-13-0": {"driven_any": 1.001633421251193, "sim_physics": 0.0938559266237112, "survival_time": 2.5999999999999988, "driven_lanedir": 0.6537162971975778, "sim_render-ego": 0.007130063497103178, "in-drivable-lane": 0.5999999999999979, "agent_compute-ego": 0.013753689252413236, "deviation-heading": 0.4928440244444201, "set_robot_commands": 0.00436552671285776, "deviation-center-line": 0.13361893417935136, "driven_lanedir_consec": 0.6537162971975778, "sim_compute_sim_state": 0.0027355689268845776, "sim_compute_performance-ego": 0.0036947910602276143, "sim_compute_robot_state-ego": 0.0049799955808199365, "sim_compute_robot_state-npc0": 0.0046492815017700195, "sim_compute_robot_state-npc1": 0.00430829249895536, "sim_compute_robot_state-npc2": 0.004288114034212553, "sim_compute_robot_state-npc3": 0.004336875218611497}, "udem1-14-0": {"driven_any": 2.218769782727205, "sim_physics": 0.08944214639209566, "survival_time": 5.249999999999989, "driven_lanedir": 0.8125963661433687, "sim_render-ego": 0.0067785649072556265, "in-drivable-lane": 2.499999999999993, "agent_compute-ego": 0.013117551803588867, "deviation-heading": 1.7163026916317137, "set_robot_commands": 0.004307483491443453, "deviation-center-line": 0.2412693316050378, "driven_lanedir_consec": 0.6077880672015876, "sim_compute_sim_state": 0.0026207855769566127, "sim_compute_performance-ego": 0.0036090555645170662, "sim_compute_robot_state-ego": 0.004701621191842215, "sim_compute_robot_state-npc0": 0.004370178495134626, "sim_compute_robot_state-npc1": 0.004239279883248465, "sim_compute_robot_state-npc2": 0.004226475670224144, "sim_compute_robot_state-npc3": 0.004220848991757347}}
set_robot_commands_max0.008506011359299286
set_robot_commands_mean0.005713724672374173
set_robot_commands_median0.004471540451049805
set_robot_commands_min0.004307483491443453
sim_compute_performance-ego_max0.006853547277329843
sim_compute_performance-ego_mean0.004638541518036289
sim_compute_performance-ego_median0.0036947910602276143
sim_compute_performance-ego_min0.0036090555645170662
sim_compute_robot_state-ego_max0.009203808217109003
sim_compute_robot_state-ego_mean0.006252789269497823
sim_compute_robot_state-ego_median0.004959604956886985
sim_compute_robot_state-ego_min0.004701621191842215
sim_compute_robot_state-npc0_max0.008433013022700442
sim_compute_robot_state-npc0_mean0.005791352941660301
sim_compute_robot_state-npc0_median0.0046492815017700195
sim_compute_robot_state-npc0_min0.004370178495134626
sim_compute_robot_state-npc1_max0.008478948048182897
sim_compute_robot_state-npc1_mean0.005586154397044653
sim_compute_robot_state-npc1_median0.004322226230914776
sim_compute_robot_state-npc1_min0.004239279883248465
sim_compute_robot_state-npc2_max0.008507061608230011
sim_compute_robot_state-npc2_mean0.005565510160406131
sim_compute_robot_state-npc2_median0.004304427366990309
sim_compute_robot_state-npc2_min0.004226475670224144
sim_compute_robot_state-npc3_max0.0081188769280156
sim_compute_robot_state-npc3_mean0.005473505841087629
sim_compute_robot_state-npc3_median0.004323666745966131
sim_compute_robot_state-npc3_min0.004220848991757347
sim_compute_sim_state_max0.005276583418061462
sim_compute_sim_state_mean0.003434525754481171
sim_compute_sim_state_median0.0027355689268845776
sim_compute_sim_state_min0.0026207855769566127
sim_physics_max0.19696012565067836
sim_physics_mean0.1250990785155662
sim_physics_median0.10486354351043702
sim_physics_min0.08944214639209566
sim_render-ego_max0.012926835048047805
sim_render-ego_mean0.008739367533562524
sim_render-ego_median0.007247740572149103
sim_render-ego_min0.0067785649072556265
simulation-passed1
survival_time_max8.349999999999984
survival_time_mean3.3333333333333286
survival_time_min1.0500000000000005
No reset possible
247654315Tanli Chou 🇹🇼challenge-aido_LF-template-randomaido3-LF-sim-validationstep1-simulationsuccessnoip-172-31-43-40-104980:05:53
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.5723310224856362
survival_time_median2.549999999999999
deviation-center-line_median0.13005513204193506
in-drivable-lane_median0.29999999999999893


other stats
agent_compute-ego_max0.018319262398613825
agent_compute-ego_mean0.01692411076402195
agent_compute-ego_median0.016927644318225337
agent_compute-ego_min0.015485440983491786
deviation-center-line_max0.2072381278624381
deviation-center-line_mean0.13189395308989765
deviation-center-line_min0.0788839760023766
deviation-heading_max0.5289022616823588
deviation-heading_mean0.3206717506075888
deviation-heading_median0.23606825392187256
deviation-heading_min0.19401296908406307
driven_any_max1.6182144577009443
driven_any_mean0.861378585933269
driven_any_median0.9933867744747804
driven_any_min0.23626560261560617
driven_lanedir_consec_max0.8010406009604816
driven_lanedir_consec_mean0.5597645764431625
driven_lanedir_consec_min0.2264965947033466
driven_lanedir_max0.8010406009604816
driven_lanedir_mean0.5597645764431625
driven_lanedir_median0.5723310224856362
driven_lanedir_min0.2264965947033466
in-drivable-lane_max1.7999999999999936
in-drivable-lane_mean0.619999999999998
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.0243289631429704, "sim_physics": 0.06530661936159488, "survival_time": 2.6999999999999984, "driven_lanedir": 0.5723310224856362, "sim_render-ego": 0.009256009702329282, "in-drivable-lane": 0.9999999999999976, "agent_compute-ego": 0.018319262398613825, "deviation-heading": 0.19401296908406307, "set_robot_commands": 0.0061068888063784, "deviation-center-line": 0.15907928712407876, "driven_lanedir_consec": 0.5723310224856362, "sim_compute_sim_state": 0.003819810019599067, "sim_compute_performance-ego": 0.005420305110790112, "sim_compute_robot_state-ego": 0.006983509770146123}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.9933867744747804, "sim_physics": 0.06708376547869514, "survival_time": 2.549999999999999, "driven_lanedir": 0.8010406009604816, "sim_render-ego": 0.00919798776215198, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.016927644318225337, "deviation-heading": 0.5289022616823588, "set_robot_commands": 0.006183138080671721, "deviation-center-line": 0.13005513204193506, "driven_lanedir_consec": 0.8010406009604816, "sim_compute_sim_state": 0.003727969001321232, "sim_compute_performance-ego": 0.004850583917954389, "sim_compute_robot_state-ego": 0.006963846730250938}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.23626560261560617, "sim_physics": 0.0581978068632238, "survival_time": 0.8500000000000002, "driven_lanedir": 0.2264965947033466, "sim_render-ego": 0.008267486796659581, "in-drivable-lane": 0, "agent_compute-ego": 0.015485440983491786, "deviation-heading": 0.23048898560984332, "set_robot_commands": 0.0052478453692267925, "deviation-center-line": 0.0788839760023766, "driven_lanedir_consec": 0.2264965947033466, "sim_compute_sim_state": 0.003395529354319853, "sim_compute_performance-ego": 0.0044230994056252875, "sim_compute_robot_state-ego": 0.005799293518066406}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.4346971317320432, "sim_physics": 0.05690665245056152, "survival_time": 1.2500000000000004, "driven_lanedir": 0.40985475638935265, "sim_render-ego": 0.008836307525634766, "in-drivable-lane": 0, "agent_compute-ego": 0.01672414779663086, "deviation-heading": 0.4138862827398061, "set_robot_commands": 0.006307411193847656, "deviation-center-line": 0.08421324241865966, "driven_lanedir_consec": 0.40985475638935265, "sim_compute_sim_state": 0.0034594821929931643, "sim_compute_performance-ego": 0.0049673843383789065, "sim_compute_robot_state-ego": 0.006081800460815429}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 1.6182144577009443, "sim_physics": 0.061351625225212, "survival_time": 3.949999999999994, "driven_lanedir": 0.7890999076769956, "sim_render-ego": 0.00883547565605067, "in-drivable-lane": 1.7999999999999936, "agent_compute-ego": 0.01716405832314793, "deviation-heading": 0.23606825392187256, "set_robot_commands": 0.006095246423648883, "deviation-center-line": 0.2072381278624381, "driven_lanedir_consec": 0.7890999076769956, "sim_compute_sim_state": 0.003787082961842983, "sim_compute_performance-ego": 0.004876806766171998, "sim_compute_robot_state-ego": 0.006223216841492472}}
set_robot_commands_max0.006307411193847656
set_robot_commands_mean0.00598810597475469
set_robot_commands_median0.0061068888063784
set_robot_commands_min0.0052478453692267925
sim_compute_performance-ego_max0.005420305110790112
sim_compute_performance-ego_mean0.004907635907784138
sim_compute_performance-ego_median0.004876806766171998
sim_compute_performance-ego_min0.0044230994056252875
sim_compute_robot_state-ego_max0.006983509770146123
sim_compute_robot_state-ego_mean0.006410333464154273
sim_compute_robot_state-ego_median0.006223216841492472
sim_compute_robot_state-ego_min0.005799293518066406
sim_compute_sim_state_max0.003819810019599067
sim_compute_sim_state_mean0.00363797470601526
sim_compute_sim_state_median0.003727969001321232
sim_compute_sim_state_min0.003395529354319853
sim_physics_max0.06708376547869514
sim_physics_mean0.061769293875857466
sim_physics_median0.061351625225212
sim_physics_min0.05690665245056152
sim_render-ego_max0.009256009702329282
sim_render-ego_mean0.008878653488565254
sim_render-ego_median0.008836307525634766
sim_render-ego_min0.008267486796659581
simulation-passed1
survival_time_max3.949999999999994
survival_time_mean2.2599999999999985
survival_time_min0.8500000000000002
No reset possible
247614310Frank (Chude) Qian 🇨🇦PredictorLastaido2-PREDstep1-simulationhost-erroryesip-172-31-43-40-104980:00:35
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 641, in get_cr
    wd, aws_config, copy_to_machine_cache=copy_to_machine_cache
  File "/project/src/duckietown_challenges_runner/runner.py", line 1301, in upload_files
    aws_config, toupload, copy_to_machine_cache=copy_to_machine_cache
  File "/project/src/duckietown_challenges_runner/runner.py", line 1555, in upload
    sha256hex = compute_sha256hex(realfile)
  File "/project/src/duckietown_challenges_runner/runner.py", line 1611, in compute_sha256hex
    res: bytes = subprocess.check_output(cmd)
  File "/usr/lib/python3.6/subprocess.py", line 356, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['shasum', '-a', '256', '/tmp/duckietown/DT18/evaluator/executions/aido2-PRED/submission4310/step1-simulation-ip-172-31-43-40-10498-job24761/logs/evaluation/stderr.html']' returned non-zero exit status 1.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
247434306Frank (Chude) Qian 🇨🇦Frank test submissionaido3-LFVI-sim-testingstep1-simulationsuccessnoip-172-31-43-40-104980:12:50
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.4541194076078226
survival_time_median2.0500000000000007
deviation-center-line_median0.1184981700628623
in-drivable-lane_median0


other stats
agent_compute-ego_max0.030442302509889763
agent_compute-ego_mean0.019273227822323047
agent_compute-ego_median0.016355931758880615
agent_compute-ego_min0.015961611270904542
deviation-center-line_max0.27212935355303575
deviation-center-line_mean0.12800192803639482
deviation-center-line_min0.054904029848945685
deviation-heading_max0.9814018754699168
deviation-heading_mean0.39865622452675337
deviation-heading_median0.2921306476839709
deviation-heading_min0.13989224355569133
driven_any_max2.7696546371463566
driven_any_mean1.0880644329743805
driven_any_median1.0296612854712663
driven_any_min0.37149985167138017
driven_lanedir_consec_max1.9242206205397283
driven_lanedir_consec_mean0.7675641832273904
driven_lanedir_consec_min0.33418991107979124
driven_lanedir_max1.924267585170248
driven_lanedir_mean0.7820426634438573
driven_lanedir_median0.618479321964448
driven_lanedir_min0.33418991107979124
in-drivable-lane_max1.899999999999998
in-drivable-lane_mean0.4266666666666661
in-drivable-lane_min0
per-episodes
details{"udem1-0-0": {"driven_any": 1.5696547037668402, "sim_physics": 0.186960070820178, "survival_time": 2.9499999999999975, "driven_lanedir": 1.5638383114510817, "sim_render-ego": 0.012356616682925469, "in-drivable-lane": 0, "agent_compute-ego": 0.030442302509889763, "deviation-heading": 0.2528567413804248, "set_robot_commands": 0.009440187680519234, "deviation-center-line": 0.19417316650714284, "driven_lanedir_consec": 1.5636399177084477, "sim_compute_sim_state": 0.0051993798401396155, "sim_compute_performance-ego": 0.006976996437977936, "sim_compute_robot_state-ego": 0.009500188342595504, "sim_compute_robot_state-npc0": 0.009075738615908864, "sim_compute_robot_state-npc1": 0.008179656529830675, "sim_compute_robot_state-npc2": 0.008295002630201437, "sim_compute_robot_state-npc3": 0.008349095360707428}, "udem1-1-0": {"driven_any": 0.46051146883057115, "sim_physics": 0.17277137799696488, "survival_time": 1.1000000000000003, "driven_lanedir": 0.43495908029179065, "sim_render-ego": 0.010632265697826038, "in-drivable-lane": 0, "agent_compute-ego": 0.02666889537464489, "deviation-heading": 0.3681646786246837, "set_robot_commands": 0.008314327760176226, "deviation-center-line": 0.07726482156623102, "driven_lanedir_consec": 0.43495908029179065, "sim_compute_sim_state": 0.004447525197809393, "sim_compute_performance-ego": 0.0059302936900745735, "sim_compute_robot_state-ego": 0.007314887913790616, "sim_compute_robot_state-npc0": 0.007741754705255682, "sim_compute_robot_state-npc1": 0.007515744729475541, "sim_compute_robot_state-npc2": 0.0073615637692538176, "sim_compute_robot_state-npc3": 0.00739885460246693}, "udem1-2-0": {"driven_any": 2.1696546375441614, "sim_physics": 0.17968391466744338, "survival_time": 3.949999999999994, "driven_lanedir": 1.7414407758715231, "sim_render-ego": 0.011681303193297568, "in-drivable-lane": 0.6999999999999975, "agent_compute-ego": 0.026797077323817, "deviation-heading": 0.2785063205851342, "set_robot_commands": 0.008345238770110697, "deviation-center-line": 0.22563902167216496, "driven_lanedir_consec": 1.7413838620089876, "sim_compute_sim_state": 0.004742211933377423, "sim_compute_performance-ego": 0.006413963776600512, "sim_compute_robot_state-ego": 0.008612617661681357, "sim_compute_robot_state-npc0": 0.007899048962170564, "sim_compute_robot_state-npc1": 0.007656592357007763, "sim_compute_robot_state-npc2": 0.007622703721251668, "sim_compute_robot_state-npc3": 0.007743159426918513}, "udem1-3-0": {"driven_any": 2.7696546371463566, "sim_physics": 0.16528369200350057, "survival_time": 4.94999999999999, "driven_lanedir": 1.924267585170248, "sim_render-ego": 0.011512086849019986, "in-drivable-lane": 1.399999999999995, "agent_compute-ego": 0.02653347843825215, "deviation-heading": 0.2652784439875202, "set_robot_commands": 0.008130470911661783, "deviation-center-line": 0.27212935355303575, "driven_lanedir_consec": 1.9242206205397283, "sim_compute_sim_state": 0.004664324750803938, "sim_compute_performance-ego": 0.006614290102563723, "sim_compute_robot_state-ego": 0.008319616317749023, "sim_compute_robot_state-npc0": 0.007910658614804047, "sim_compute_robot_state-npc1": 0.007959758392488114, "sim_compute_robot_state-npc2": 0.0077899128499657215, "sim_compute_robot_state-npc3": 0.00772158545677108}, "udem1-4-0": {"driven_any": 0.4010834948810282, "sim_physics": 0.0945006251335144, "survival_time": 1.0000000000000002, "driven_lanedir": 0.38720104799654376, "sim_render-ego": 0.006947660446166992, "in-drivable-lane": 0, "agent_compute-ego": 0.015961611270904542, "deviation-heading": 0.2638670899443274, "set_robot_commands": 0.004644072055816651, "deviation-center-line": 0.060733287044270105, "driven_lanedir_consec": 0.38720104799654376, "sim_compute_sim_state": 0.0027395248413085937, "sim_compute_performance-ego": 0.0036797404289245607, "sim_compute_robot_state-ego": 0.004835188388824463, "sim_compute_robot_state-npc0": 0.004657816886901855, "sim_compute_robot_state-npc1": 0.004505264759063721, "sim_compute_robot_state-npc2": 0.0043667078018188475, "sim_compute_robot_state-npc3": 0.004347157478332519}, "udem1-5-0": {"driven_any": 1.0596597849416796, "sim_physics": 0.09307186376480828, "survival_time": 2.1000000000000005, "driven_lanedir": 0.419433267065787, "sim_render-ego": 0.006911277770996094, "in-drivable-lane": 0.8500000000000006, "agent_compute-ego": 0.016063184965224492, "deviation-heading": 0.7002960066861377, "set_robot_commands": 0.004330390975588844, "deviation-center-line": 0.1184981700628623, "driven_lanedir_consec": 0.419433267065787, "sim_compute_sim_state": 0.0026538939703078497, "sim_compute_performance-ego": 0.0037105651128859743, "sim_compute_robot_state-ego": 0.0047471920649210615, "sim_compute_robot_state-npc0": 0.0044436170941307435, "sim_compute_robot_state-npc1": 0.0042776720864432195, "sim_compute_robot_state-npc2": 0.004256271180652436, "sim_compute_robot_state-npc3": 0.004233155931745257}, "udem1-6-0": {"driven_any": 0.8796785144013353, "sim_physics": 0.11215579509735107, "survival_time": 1.800000000000001, "driven_lanedir": 0.8770246919234508, "sim_render-ego": 0.007030195660061306, "in-drivable-lane": 0, "agent_compute-ego": 0.016259200043148465, "deviation-heading": 0.13989224355569133, "set_robot_commands": 0.004412107997470432, "deviation-center-line": 0.15455097642444732, "driven_lanedir_consec": 0.8770246919234508, "sim_compute_sim_state": 0.0027115411228603786, "sim_compute_performance-ego": 0.0036084983083936903, "sim_compute_robot_state-ego": 0.00469689236746894, "sim_compute_robot_state-npc0": 0.00448155403137207, "sim_compute_robot_state-npc1": 0.0043012963400946725, "sim_compute_robot_state-npc2": 0.004272719224294026, "sim_compute_robot_state-npc3": 0.004255712032318115}, "udem1-7-0": {"driven_any": 1.0296612854712832, "sim_physics": 0.10663809427400917, "survival_time": 2.0500000000000007, "driven_lanedir": 0.8977146610328521, "sim_render-ego": 0.006922512519650343, "in-drivable-lane": 0, "agent_compute-ego": 0.016618775158393675, "deviation-heading": 0.7301693312178688, "set_robot_commands": 0.004385081733145365, "deviation-center-line": 0.12694367940927642, "driven_lanedir_consec": 0.8807130056354433, "sim_compute_sim_state": 0.0027237461834419066, "sim_compute_performance-ego": 0.003634929656982422, "sim_compute_robot_state-ego": 0.004777809468711295, "sim_compute_robot_state-npc0": 0.004526795410528415, "sim_compute_robot_state-npc1": 0.004293592964730612, "sim_compute_robot_state-npc2": 0.004277101377161537, "sim_compute_robot_state-npc3": 0.004239105596774962}, "udem1-8-0": {"driven_any": 1.0296612854712663, "sim_physics": 0.09062839135891054, "survival_time": 2.0500000000000007, "driven_lanedir": 0.618479321964448, "sim_render-ego": 0.006886522944380597, "in-drivable-lane": 0.5000000000000004, "agent_compute-ego": 0.016115514243521343, "deviation-heading": 0.6847850662034057, "set_robot_commands": 0.004331867869307355, "deviation-center-line": 0.13317992938688095, "driven_lanedir_consec": 0.618479321964448, "sim_compute_sim_state": 0.002654697836899176, "sim_compute_performance-ego": 0.003632981602738543, "sim_compute_robot_state-ego": 0.004653395676031345, "sim_compute_robot_state-npc0": 0.004465940521984566, "sim_compute_robot_state-npc1": 0.00426084239308427, "sim_compute_robot_state-npc2": 0.004241181582939334, "sim_compute_robot_state-npc3": 0.004227771991636695}, "udem1-9-0": {"driven_any": 0.4605114688305781, "sim_physics": 0.1112384471026334, "survival_time": 1.1000000000000003, "driven_lanedir": 0.4443672320455454, "sim_render-ego": 0.0072012489492242985, "in-drivable-lane": 0, "agent_compute-ego": 0.01672438057986173, "deviation-heading": 0.2921306476839709, "set_robot_commands": 0.004176269878040661, "deviation-center-line": 0.06393804025783281, "driven_lanedir_consec": 0.4443672320455454, "sim_compute_sim_state": 0.002665909853848544, "sim_compute_performance-ego": 0.003672686490145597, "sim_compute_robot_state-ego": 0.004800113764676181, "sim_compute_robot_state-npc0": 0.004521835934032093, "sim_compute_robot_state-npc1": 0.004211490804498846, "sim_compute_robot_state-npc2": 0.004188580946488814, "sim_compute_robot_state-npc3": 0.004186966202475808}, "udem1-10-0": {"driven_any": 0.37149985167138017, "sim_physics": 0.09747221595362614, "survival_time": 0.9500000000000004, "driven_lanedir": 0.3517991142132264, "sim_render-ego": 0.007022418473896228, "in-drivable-lane": 0, "agent_compute-ego": 0.016446352005004883, "deviation-heading": 0.31076793256366336, "set_robot_commands": 0.004469407232184159, "deviation-center-line": 0.054904029848945685, "driven_lanedir_consec": 0.3517991142132264, "sim_compute_sim_state": 0.0026927872707969264, "sim_compute_performance-ego": 0.003665673105340255, "sim_compute_robot_state-ego": 0.004726949490998921, "sim_compute_robot_state-npc0": 0.004514982825831363, "sim_compute_robot_state-npc1": 0.004392448224519428, "sim_compute_robot_state-npc2": 0.004338076240138004, "sim_compute_robot_state-npc3": 0.004322829999421772}, "udem1-11-0": {"driven_any": 0.4307612897755775, "sim_physics": 0.0908636933281308, "survival_time": 1.0500000000000005, "driven_lanedir": 0.4261291824030562, "sim_render-ego": 0.007025253205072312, "in-drivable-lane": 0, "agent_compute-ego": 0.016103869392758326, "deviation-heading": 0.1541202345559414, "set_robot_commands": 0.004376218432471866, "deviation-center-line": 0.09941469755038562, "driven_lanedir_consec": 0.4261291824030562, "sim_compute_sim_state": 0.0026623294467017764, "sim_compute_performance-ego": 0.003633306139991397, "sim_compute_robot_state-ego": 0.004659050986880348, "sim_compute_robot_state-npc0": 0.004480940955025809, "sim_compute_robot_state-npc1": 0.004287606193905785, "sim_compute_robot_state-npc2": 0.004267090842837379, "sim_compute_robot_state-npc3": 0.004276355107625325}, "udem1-12-0": {"driven_any": 1.4796547806284426, "sim_physics": 0.09766509703227452, "survival_time": 2.799999999999998, "driven_lanedir": 0.33418991107979124, "sim_render-ego": 0.006970213992255074, "in-drivable-lane": 1.899999999999998, "agent_compute-ego": 0.016355931758880615, "deviation-heading": 0.193170346576385, "set_robot_commands": 0.004452564886638096, "deviation-center-line": 0.09687090955952632, "driven_lanedir_consec": 0.33418991107979124, "sim_compute_sim_state": 0.002733175243650164, "sim_compute_performance-ego": 0.0036292203835078646, "sim_compute_robot_state-ego": 0.004781855004174369, "sim_compute_robot_state-npc0": 0.004590417657579694, "sim_compute_robot_state-npc1": 0.004314452409744263, "sim_compute_robot_state-npc2": 0.004291662148066929, "sim_compute_robot_state-npc3": 0.004291198083332607}, "udem1-13-0": {"driven_any": 0.9996632207705788, "sim_physics": 0.08940652012825012, "survival_time": 2.000000000000001, "driven_lanedir": 0.655803085926784, "sim_render-ego": 0.006929236650466919, "in-drivable-lane": 0.4500000000000002, "agent_compute-ego": 0.015985780954360963, "deviation-heading": 0.36443640886622947, "set_robot_commands": 0.004356217384338379, "deviation-center-line": 0.1026590609494026, "driven_lanedir_consec": 0.655803085926784, "sim_compute_sim_state": 0.002657592296600342, "sim_compute_performance-ego": 0.003612416982650757, "sim_compute_robot_state-ego": 0.004654616117477417, "sim_compute_robot_state-npc0": 0.0044558167457580565, "sim_compute_robot_state-npc1": 0.004260820150375366, "sim_compute_robot_state-npc2": 0.004253017902374268, "sim_compute_robot_state-npc3": 0.004253947734832763}, "udem1-14-0": {"driven_any": 1.2096560704846229, "sim_physics": 0.0961539035147809, "survival_time": 2.3499999999999996, "driven_lanedir": 0.6539926832217301, "sim_render-ego": 0.006880024646190887, "in-drivable-lane": 0.5999999999999994, "agent_compute-ego": 0.01602206331618289, "deviation-heading": 0.9814018754699168, "set_robot_commands": 0.004435934918992063, "deviation-center-line": 0.1391297767535178, "driven_lanedir_consec": 0.4541194076078226, "sim_compute_sim_state": 0.0027525729321418923, "sim_compute_performance-ego": 0.003632317198083756, "sim_compute_robot_state-ego": 0.004786268193671044, "sim_compute_robot_state-npc0": 0.004586052387318713, "sim_compute_robot_state-npc1": 0.00429565855797301, "sim_compute_robot_state-npc2": 0.004305124282836914, "sim_compute_robot_state-npc3": 0.0043207432361359295}}
set_robot_commands_max0.009440187680519234
set_robot_commands_mean0.005506690565764121
set_robot_commands_median0.004435934918992063
set_robot_commands_min0.004176269878040661
sim_compute_performance-ego_max0.006976996437977936
sim_compute_performance-ego_mean0.004403191961124104
sim_compute_performance-ego_median0.003665673105340255
sim_compute_performance-ego_min0.0036084983083936903
sim_compute_robot_state-ego_max0.009500188342595504
sim_compute_robot_state-ego_mean0.005724442783976793
sim_compute_robot_state-ego_median0.004781855004174369
sim_compute_robot_state-ego_min0.004653395676031345
sim_compute_robot_state-npc0_max0.009075738615908864
sim_compute_robot_state-npc0_mean0.005490198089906836
sim_compute_robot_state-npc0_median0.004526795410528415
sim_compute_robot_state-npc0_min0.0044436170941307435
sim_compute_robot_state-npc1_max0.008179656529830675
sim_compute_robot_state-npc1_mean0.005247526459549018
sim_compute_robot_state-npc1_median0.0043012963400946725
sim_compute_robot_state-npc1_min0.004211490804498846
sim_compute_robot_state-npc2_max0.008295002630201437
sim_compute_robot_state-npc2_mean0.00520844776668541
sim_compute_robot_state-npc2_median0.004291662148066929
sim_compute_robot_state-npc2_min0.004188580946488814
sim_compute_robot_state-npc3_max0.008349095360707428
sim_compute_robot_state-npc3_mean0.00521117588276638
sim_compute_robot_state-npc3_median0.004291198083332607
sim_compute_robot_state-npc3_min0.004186966202475808
sim_compute_sim_state_max0.0051993798401396155
sim_compute_sim_state_mean0.003246747514712528
sim_compute_sim_state_median0.0027237461834419066
sim_compute_sim_state_min0.0026538939703078497
sim_physics_max0.186960070820178
sim_physics_mean0.1189662468117584
sim_physics_median0.09766509703227452
sim_physics_min0.08940652012825012
sim_render-ego_max0.012356616682925469
sim_render-ego_mean0.008193922512095341
sim_render-ego_median0.007022418473896228
sim_render-ego_min0.006880024646190887
simulation-passed1
survival_time_max4.94999999999999
survival_time_mean2.1466666666666656
survival_time_min0.9500000000000004
No reset possible
247373257Liu Sam 🇹🇼PredictorLastaido2-PREDstep1-simulationhost-erroryesip-172-31-43-40-104980:00:05
Error while running [...]

stderr | Pulling solution   ... 
stderr | Pulling evaluation ... done
stderr | 
stderr | ERROR: for solution  pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr | pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr |
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
247313264Liu Sam 🇹🇼challenge-aido_LF-template-randomaido2-PREDstep1-simulationhost-erroryesip-172-31-43-40-104980:00:04
Error while running [...]

stderr | Pulling solution   ... 
stderr | Pulling evaluation ... done
stderr | 
stderr | ERROR: for solution  pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr | pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr |
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
247273265Liu Sam 🇹🇼challenge-aido_LF-template-randomaido2-PREDstep1-simulationhost-erroryesip-172-31-43-40-104980:00:06
Error while running [...]

stderr | Pulling solution   ... 
stderr | Pulling evaluation ... done
stderr | 
stderr | ERROR: for solution  pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr | pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr |
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
247162886Diego Charrez 🇵🇪PredictorLastaido2-PREDstep1-simulationsuccessyesip-172-31-43-40-104980:00:20
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L10.1052849061482706
error_L20.01996025034131507


No reset possible
247032913Claudio RuchPredictorLastaido2-PREDstep1-simulationsuccessyesip-172-31-43-40-104980:00:30
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L10.11698519906101312
error_L20.02197187043780394


No reset possible
246933165Jui-Te Huang 🇹🇼PredictorLastaido2-PREDstep1-simulationsuccessyesip-172-31-43-40-104980:00:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L10.12002118350744206
error_L20.0215184485057129


No reset possible
246813180ZongRu LiPredictorLastaido2-PREDstep1-simulationsuccessyesip-172-31-43-40-104980:00:15
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L10.1097258796481436
error_L20.019528154625895


No reset possible
246703181Pokai ChangPredictorLastaido2-PREDstep1-simulationsuccessyesip-172-31-43-40-104980:00:15
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L10.10962913938456646
error_L20.019115036553287692


No reset possible
246583202Masaya Kambara 🇯🇵PredictorRandomaido2-PREDstep1-simulationsuccessyesip-172-31-43-40-104980:00:43
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L10.306387608792548
error_L20.1443631968325346


No reset possible
245594293Liam Paull 🇨🇦challenge-aido_LF-template-randomaido3-LF-sim-testingstep1-simulationerrornoip-172-31-43-40-104980:08:05
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
245344117Xiao'ao Song 🇨🇭PredictorLastaido2-PREDstep1-simulationsuccessyesip-172-31-43-40-104980:01:24
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L10.1127948807166666
error_L20.01883081948397549


No reset possible
245274288Gianmarco BernasconiBaseline-IL-logs-tensorflowaido3-LF-sim-validationstep1-simulationsuccessnoip-172-31-43-40-104980:10:27
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median-0.04162293675663564
survival_time_median14.950000000000076
deviation-center-line_median0.7646614942498801
in-drivable-lane_median0


other stats
agent_compute-ego_max0.04945469538370768
agent_compute-ego_mean0.04853820816675822
agent_compute-ego_median0.048401366074879965
agent_compute-ego_min0.047729369004567465
deviation-center-line_max1.446985045231216
deviation-center-line_mean0.8718261383642056
deviation-center-line_min0.3977555450527604
deviation-heading_max5.841920887817183
deviation-heading_mean4.319367319373697
deviation-heading_median4.75827418670767
deviation-heading_min1.709756693534375
driven_any_max0.2609349036469851
driven_any_mean0.18277967343845697
driven_any_median0.21242998352179285
driven_any_min0.05488199361760352
driven_lanedir_consec_max0.07250820236920275
driven_lanedir_consec_mean-0.040300647213044424
driven_lanedir_consec_min-0.14921282064119137
driven_lanedir_max0.07250820236920275
driven_lanedir_mean-0.040300647213044424
driven_lanedir_median-0.04162293675663564
driven_lanedir_min-0.14921282064119137
in-drivable-lane_max7.100000000000056
in-drivable-lane_mean1.420000000000011
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.2609349036469851, "sim_physics": 0.05165888786315918, "survival_time": 14.950000000000076, "driven_lanedir": -0.10250190344551148, "sim_render-ego": 0.007010696729024251, "in-drivable-lane": 7.100000000000056, "agent_compute-ego": 0.04945469538370768, "deviation-heading": 5.841920887817183, "set_robot_commands": 0.0044779348373413085, "deviation-center-line": 1.0192916365957454, "driven_lanedir_consec": -0.10250190344551148, "sim_compute_sim_state": 0.0028341197967529297, "sim_compute_performance-ego": 0.0037661051750183102, "sim_compute_robot_state-ego": 0.004818933010101318}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.2225709688706906, "sim_physics": 0.04608408212661743, "survival_time": 14.950000000000076, "driven_lanedir": 0.019326222408913577, "sim_render-ego": 0.006805244286855062, "in-drivable-lane": 0, "agent_compute-ego": 0.04804979642232259, "deviation-heading": 4.062579191786069, "set_robot_commands": 0.0044740891456604, "deviation-center-line": 0.3977555450527604, "driven_lanedir_consec": 0.019326222408913577, "sim_compute_sim_state": 0.0028134687741597493, "sim_compute_performance-ego": 0.0036751627922058105, "sim_compute_robot_state-ego": 0.004725116888682048}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.05488199361760352, "sim_physics": 0.04632042566935221, "survival_time": 14.950000000000076, "driven_lanedir": -0.04162293675663564, "sim_render-ego": 0.006741500695546468, "in-drivable-lane": 0, "agent_compute-ego": 0.0490558139483134, "deviation-heading": 4.75827418670767, "set_robot_commands": 0.0044957780838012695, "deviation-center-line": 0.7304369706914263, "driven_lanedir_consec": -0.04162293675663564, "sim_compute_sim_state": 0.002860533396402995, "sim_compute_performance-ego": 0.0037051701545715338, "sim_compute_robot_state-ego": 0.004816596508026123}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.16308051753521258, "sim_physics": 0.047319008509318035, "survival_time": 14.950000000000076, "driven_lanedir": -0.14921282064119137, "sim_render-ego": 0.00675984779993693, "in-drivable-lane": 0, "agent_compute-ego": 0.047729369004567465, "deviation-heading": 5.224305637023191, "set_robot_commands": 0.004468513329823812, "deviation-center-line": 1.446985045231216, "driven_lanedir_consec": -0.14921282064119137, "sim_compute_sim_state": 0.0028248333930969236, "sim_compute_performance-ego": 0.003682561715443929, "sim_compute_robot_state-ego": 0.004744172890981039}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.21242998352179285, "sim_physics": 0.04837456146876017, "survival_time": 14.950000000000076, "driven_lanedir": 0.07250820236920275, "sim_render-ego": 0.006810235182444255, "in-drivable-lane": 0, "agent_compute-ego": 0.048401366074879965, "deviation-heading": 1.709756693534375, "set_robot_commands": 0.004462720553080241, "deviation-center-line": 0.7646614942498801, "driven_lanedir_consec": 0.07250820236920275, "sim_compute_sim_state": 0.0028459016482035317, "sim_compute_performance-ego": 0.003693061669667562, "sim_compute_robot_state-ego": 0.00480215311050415}}
set_robot_commands_max0.0044957780838012695
set_robot_commands_mean0.004475807189941406
set_robot_commands_median0.0044740891456604
set_robot_commands_min0.004462720553080241
sim_compute_performance-ego_max0.0037661051750183102
sim_compute_performance-ego_mean0.0037044123013814296
sim_compute_performance-ego_median0.003693061669667562
sim_compute_performance-ego_min0.0036751627922058105
sim_compute_robot_state-ego_max0.004818933010101318
sim_compute_robot_state-ego_mean0.004781394481658936
sim_compute_robot_state-ego_median0.00480215311050415
sim_compute_robot_state-ego_min0.004725116888682048
sim_compute_sim_state_max0.002860533396402995
sim_compute_sim_state_mean0.002835771401723226
sim_compute_sim_state_median0.0028341197967529297
sim_compute_sim_state_min0.0028134687741597493
sim_physics_max0.05165888786315918
sim_physics_mean0.047951393127441407
sim_physics_median0.047319008509318035
sim_physics_min0.04608408212661743
sim_render-ego_max0.007010696729024251
sim_render-ego_mean0.006825504938761393
sim_render-ego_median0.006805244286855062
sim_render-ego_min0.006741500695546468
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean14.950000000000076
survival_time_min14.950000000000076
No reset possible
245254287Liam Paull 🇨🇦challenge-aido_LF-template-randomaido3-LFVI-sim-validationstep1-simulationsuccessnoip-172-31-43-40-104980:06:42
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.493517638549009
survival_time_median2.3999999999999995
deviation-center-line_median0.09791133281082
in-drivable-lane_median0.549999999999998


other stats
agent_compute-ego_max0.018602959717376324
agent_compute-ego_mean0.01685407203985667
agent_compute-ego_median0.016675052188691638
agent_compute-ego_min0.014300952355066936
deviation-center-line_max0.2348196785686717
deviation-center-line_mean0.1354748199226606
deviation-center-line_min0.06216029860587694
deviation-heading_max0.8115272458314912
deviation-heading_mean0.49959867397150814
deviation-heading_median0.46504862964833654
deviation-heading_min0.30044835691402494
driven_any_max2.1864003300000703
driven_any_mean1.1068208197848537
driven_any_median0.9410513078477728
driven_any_min0.34749292376309954
driven_lanedir_consec_max1.8262319453635671
driven_lanedir_consec_mean0.8047440134491037
driven_lanedir_consec_min0.3325249708009643
driven_lanedir_max1.8262319453635671
driven_lanedir_mean0.8048281473204828
driven_lanedir_median0.493517638549009
driven_lanedir_min0.3325249708009643
in-drivable-lane_max1.3499999999999952
in-drivable-lane_mean0.5599999999999984
in-drivable-lane_min0
per-episodes
details{"udem1-0-0": {"driven_any": 1.5811782993597916, "sim_physics": 0.12890202184266683, "survival_time": 3.949999999999994, "driven_lanedir": 0.9217908232230632, "sim_render-ego": 0.00969168807886824, "in-drivable-lane": 1.3499999999999952, "agent_compute-ego": 0.018602959717376324, "deviation-heading": 0.5694288623982039, "set_robot_commands": 0.0067389826231364965, "deviation-center-line": 0.2348196785686717, "driven_lanedir_consec": 0.9213701538661684, "sim_compute_sim_state": 0.0037344075456450256, "sim_compute_performance-ego": 0.005265453193761125, "sim_compute_robot_state-ego": 0.006916842883146262, "sim_compute_robot_state-npc0": 0.00637301915808569, "sim_compute_robot_state-npc1": 0.006024946140337594, "sim_compute_robot_state-npc2": 0.0061885254292548455, "sim_compute_robot_state-npc3": 0.005988235715069348}, "udem1-1-0": {"driven_any": 0.47798123795353264, "sim_physics": 0.1082641567502703, "survival_time": 1.4000000000000006, "driven_lanedir": 0.4500753586658095, "sim_render-ego": 0.007308219160352435, "in-drivable-lane": 0, "agent_compute-ego": 0.01814021383013044, "deviation-heading": 0.46504862964833654, "set_robot_commands": 0.005119894232068743, "deviation-center-line": 0.09791133281082, "driven_lanedir_consec": 0.4500753586658095, "sim_compute_sim_state": 0.00290283135005406, "sim_compute_performance-ego": 0.003797258649553571, "sim_compute_robot_state-ego": 0.005146307604653495, "sim_compute_robot_state-npc0": 0.0048989227839878625, "sim_compute_robot_state-npc1": 0.004577704838344029, "sim_compute_robot_state-npc2": 0.004564387457711356, "sim_compute_robot_state-npc3": 0.004603981971740723}, "udem1-2-0": {"driven_any": 2.1864003300000703, "sim_physics": 0.12602737111952697, "survival_time": 5.14999999999999, "driven_lanedir": 1.8262319453635671, "sim_render-ego": 0.008908315769677023, "in-drivable-lane": 0.549999999999998, "agent_compute-ego": 0.016551182108018005, "deviation-heading": 0.8115272458314912, "set_robot_commands": 0.006443854674552251, "deviation-center-line": 0.19273160416094068, "driven_lanedir_consec": 1.8262319453635671, "sim_compute_sim_state": 0.003413677215576172, "sim_compute_performance-ego": 0.004788595495872127, "sim_compute_robot_state-ego": 0.0062435622354155606, "sim_compute_robot_state-npc0": 0.005708344931741363, "sim_compute_robot_state-npc1": 0.005518279029327689, "sim_compute_robot_state-npc2": 0.005522336774659388, "sim_compute_robot_state-npc3": 0.005451051934251508}, "udem1-3-0": {"driven_any": 0.9410513078477728, "sim_physics": 0.1122071792682012, "survival_time": 2.3999999999999995, "driven_lanedir": 0.493517638549009, "sim_render-ego": 0.0076990822950998945, "in-drivable-lane": 0.8999999999999988, "agent_compute-ego": 0.014300952355066936, "deviation-heading": 0.35154027506548413, "set_robot_commands": 0.005169019103050232, "deviation-center-line": 0.08975118546699376, "driven_lanedir_consec": 0.493517638549009, "sim_compute_sim_state": 0.0030313382546106973, "sim_compute_performance-ego": 0.004012554883956909, "sim_compute_robot_state-ego": 0.005319744348526001, "sim_compute_robot_state-npc0": 0.0050052255392074585, "sim_compute_robot_state-npc1": 0.004884093999862671, "sim_compute_robot_state-npc2": 0.004789387186368306, "sim_compute_robot_state-npc3": 0.004705856243769328}, "udem1-4-0": {"driven_any": 0.34749292376309954, "sim_physics": 0.11442598842439196, "survival_time": 1.0500000000000005, "driven_lanedir": 0.3325249708009643, "sim_render-ego": 0.008836394264584496, "in-drivable-lane": 0, "agent_compute-ego": 0.016675052188691638, "deviation-heading": 0.30044835691402494, "set_robot_commands": 0.0061670030866350445, "deviation-center-line": 0.06216029860587694, "driven_lanedir_consec": 0.3325249708009643, "sim_compute_sim_state": 0.0034788336072649273, "sim_compute_performance-ego": 0.004723106111798968, "sim_compute_robot_state-ego": 0.006325063251313709, "sim_compute_robot_state-npc0": 0.005726246606735956, "sim_compute_robot_state-npc1": 0.005586885270618257, "sim_compute_robot_state-npc2": 0.005597489220755441, "sim_compute_robot_state-npc3": 0.005679584684826079}}
set_robot_commands_max0.0067389826231364965
set_robot_commands_mean0.005927750743888553
set_robot_commands_median0.0061670030866350445
set_robot_commands_min0.005119894232068743
sim_compute_performance-ego_max0.005265453193761125
sim_compute_performance-ego_mean0.0045173936669885405
sim_compute_performance-ego_median0.004723106111798968
sim_compute_performance-ego_min0.003797258649553571
sim_compute_robot_state-ego_max0.006916842883146262
sim_compute_robot_state-ego_mean0.005990304064611005
sim_compute_robot_state-ego_median0.0062435622354155606
sim_compute_robot_state-ego_min0.005146307604653495
sim_compute_robot_state-npc0_max0.00637301915808569
sim_compute_robot_state-npc0_mean0.005542351803951666
sim_compute_robot_state-npc0_median0.005708344931741363
sim_compute_robot_state-npc0_min0.0048989227839878625
sim_compute_robot_state-npc1_max0.006024946140337594
sim_compute_robot_state-npc1_mean0.0053183818556980485
sim_compute_robot_state-npc1_median0.005518279029327689
sim_compute_robot_state-npc1_min0.004577704838344029
sim_compute_robot_state-npc2_max0.0061885254292548455
sim_compute_robot_state-npc2_mean0.005332425213749867
sim_compute_robot_state-npc2_median0.005522336774659388
sim_compute_robot_state-npc2_min0.004564387457711356
sim_compute_robot_state-npc3_max0.005988235715069348
sim_compute_robot_state-npc3_mean0.005285742109931398
sim_compute_robot_state-npc3_median0.005451051934251508
sim_compute_robot_state-npc3_min0.004603981971740723
sim_compute_sim_state_max0.0037344075456450256
sim_compute_sim_state_mean0.0033122175946301764
sim_compute_sim_state_median0.003413677215576172
sim_compute_sim_state_min0.00290283135005406
sim_physics_max0.12890202184266683
sim_physics_mean0.11796534348101144
sim_physics_median0.11442598842439196
sim_physics_min0.1082641567502703
sim_render-ego_max0.00969168807886824
sim_render-ego_mean0.008488739913716417
sim_render-ego_median0.008836394264584496
sim_render-ego_min0.007308219160352435
simulation-passed1
survival_time_max5.14999999999999
survival_time_mean2.789999999999997
survival_time_min1.0500000000000005
No reset possible
245004272Riza Velioglu 🇩🇪Uni Bielefeldaido3-LFV-sim-testingstep1-simulationerrornoip-172-31-43-40-104980:01:50
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
244944261Gianmarco Bernasconichallenge-aido_LF-baseline-duckietownaido3-LF-sim-testingstep1-simulationerrornoip-172-31-43-40-104980:06:07
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
244844257Anastasiya Nikolskaya 🇷🇺betaaido3-LF-sim-validationstep1-simulationsuccessnoip-172-31-43-40-104980:09:11
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.7935366913641556
survival_time_median11.200000000000024
deviation-center-line_median0.45958033889819494
in-drivable-lane_median1.8500000000000263


other stats
agent_compute-ego_max0.053537029208558976
agent_compute-ego_mean0.050101117571273515
agent_compute-ego_median0.04912964938438102
agent_compute-ego_min0.04877588334290878
deviation-center-line_max1.026999847509937
deviation-center-line_mean0.5583911584623505
deviation-center-line_min0.34530776464833984
deviation-heading_max3.9494829628944665
deviation-heading_mean1.7942060299051965
deviation-heading_median1.263401161358298
deviation-heading_min0.9486706316682224
driven_any_max1.6883291047637146
driven_any_mean1.3109301571902423
driven_any_median1.1997999458678552
driven_any_min1.0031509963275431
driven_lanedir_consec_max1.2264511553417283
driven_lanedir_consec_mean0.8957806508801884
driven_lanedir_consec_min0.6278482467512303
driven_lanedir_max1.2264511553417283
driven_lanedir_mean0.90086989613802
driven_lanedir_median0.7935366913641556
driven_lanedir_min0.628924299030988
in-drivable-lane_max4.40000000000004
in-drivable-lane_mean2.3500000000000165
in-drivable-lane_min0.9499999999999966
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.074457552764884, "sim_physics": 0.046844499761408025, "survival_time": 8.249999999999982, "driven_lanedir": 0.628924299030988, "sim_render-ego": 0.0069646242893103396, "in-drivable-lane": 2.7999999999999936, "agent_compute-ego": 0.053537029208558976, "deviation-heading": 1.1459665517959026, "set_robot_commands": 0.004656900059093128, "deviation-center-line": 0.3490420594132677, "driven_lanedir_consec": 0.6278482467512303, "sim_compute_sim_state": 0.002956234325062145, "sim_compute_performance-ego": 0.003893876798225172, "sim_compute_robot_state-ego": 0.00501368118054939}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 1.0031509963275431, "sim_physics": 0.046782537682415685, "survival_time": 7.299999999999982, "driven_lanedir": 0.7935366913641556, "sim_render-ego": 0.0069584977136899345, "in-drivable-lane": 0.9499999999999966, "agent_compute-ego": 0.04912964938438102, "deviation-heading": 1.263401161358298, "set_robot_commands": 0.004532386178839697, "deviation-center-line": 0.34530776464833984, "driven_lanedir_consec": 0.7935366913641556, "sim_compute_sim_state": 0.0029156159048211085, "sim_compute_performance-ego": 0.003732271390418484, "sim_compute_robot_state-ego": 0.004950624622710764}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 1.1997999458678552, "sim_physics": 0.0469034058707101, "survival_time": 11.200000000000024, "driven_lanedir": 0.6300331794350629, "sim_render-ego": 0.006909745080130441, "in-drivable-lane": 4.40000000000004, "agent_compute-ego": 0.05001938449484961, "deviation-heading": 1.663508841809093, "set_robot_commands": 0.004461570509842464, "deviation-center-line": 0.6110257818420133, "driven_lanedir_consec": 0.6300331794350629, "sim_compute_sim_state": 0.002869434654712677, "sim_compute_performance-ego": 0.003693043121269771, "sim_compute_robot_state-ego": 0.004875314022813525}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 1.6883291047637146, "sim_physics": 0.047960033640265465, "survival_time": 12.800000000000049, "driven_lanedir": 1.2254041555181647, "sim_render-ego": 0.006878652609884739, "in-drivable-lane": 1.7500000000000249, "agent_compute-ego": 0.04904364142566919, "deviation-heading": 3.9494829628944665, "set_robot_commands": 0.004445907659828663, "deviation-center-line": 1.026999847509937, "driven_lanedir_consec": 1.2010339815087647, "sim_compute_sim_state": 0.0028630606830120087, "sim_compute_performance-ego": 0.0036838343366980553, "sim_compute_robot_state-ego": 0.004855738021433353}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 1.5889131862272152, "sim_physics": 0.046108340180438495, "survival_time": 11.500000000000028, "driven_lanedir": 1.2264511553417283, "sim_render-ego": 0.006883282246796981, "in-drivable-lane": 1.8500000000000263, "agent_compute-ego": 0.04877588334290878, "deviation-heading": 0.9486706316682224, "set_robot_commands": 0.004524262055106785, "deviation-center-line": 0.45958033889819494, "driven_lanedir_consec": 1.2264511553417283, "sim_compute_sim_state": 0.002855805728746497, "sim_compute_performance-ego": 0.003683781623840332, "sim_compute_robot_state-ego": 0.004813112383303435}}
set_robot_commands_max0.004656900059093128
set_robot_commands_mean0.0045242052925421475
set_robot_commands_median0.004524262055106785
set_robot_commands_min0.004445907659828663
sim_compute_performance-ego_max0.003893876798225172
sim_compute_performance-ego_mean0.003737361454090363
sim_compute_performance-ego_median0.003693043121269771
sim_compute_performance-ego_min0.003683781623840332
sim_compute_robot_state-ego_max0.00501368118054939
sim_compute_robot_state-ego_mean0.004901694046162094
sim_compute_robot_state-ego_median0.004875314022813525
sim_compute_robot_state-ego_min0.004813112383303435
sim_compute_sim_state_max0.002956234325062145
sim_compute_sim_state_mean0.002892030259270887
sim_compute_sim_state_median0.002869434654712677
sim_compute_sim_state_min0.002855805728746497
sim_physics_max0.047960033640265465
sim_physics_mean0.04691976342704755
sim_physics_median0.046844499761408025
sim_physics_min0.046108340180438495
sim_render-ego_max0.0069646242893103396
sim_render-ego_mean0.006918960387962488
sim_render-ego_median0.006909745080130441
sim_render-ego_min0.006878652609884739
simulation-passed1
survival_time_max12.800000000000049
survival_time_mean10.210000000000012
survival_time_min7.299999999999982
No reset possible
244444239Manfred DiazBaseline-IL-logs-tensorflowaido3-LF-sim-testingstep1-simulationerrornoip-172-31-43-40-104980:04:51
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
244284226Manfred DiazBaseline-IL-logs-tensorflowaido3-LFV-sim-validationstep1-simulationfailednoip-172-31-43-40-104980:00:58
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "experiment_manager.py", line 299, in run_episode
    r: MsgReceived = agent.write_topic_and_expect('get_commands', expect='commands')
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 96, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 177, in read_one
    nickname=self.nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 258, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 286, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "agent" aborted:

error in agent |Exception while handling a message on topic "get_commands".
               |
               || Traceback (most recent call last):
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1334, in _do_call
               ||     return fn(*args)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
               ||     options, feed_dict, fetch_list, target_list, run_metadata)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
               ||     run_metadata)
               || tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value prefix/ConvNet/fc_layer_2/bias
               || 	 [[{{node prefix/ConvNet/fc_layer_2/bias/read}}]]
               ||
               || During handling of the above exception, another exception occurred:
               ||
               || Traceback (most recent call last):
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 277, in loop
               ||     handle_message_node(parsed, receiver0, context0)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 431, in handle_message_node
               ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/utils.py", line 19, in call_if_fun_exists
               ||     f(**kwargs)
               ||   File "imitation_agent.py", line 71, in on_received_get_commands
               ||     pwm_left, pwm_right = self.compute_action(self.config.current_image)
               ||   File "imitation_agent.py", line 60, in compute_action
               ||     x: observation
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 929, in run
               ||     run_metadata_ptr)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1152, in _run
               ||     feed_dict_tensor, options, run_metadata)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1328, in _do_run
               ||     run_metadata)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1348, in _do_call
               ||     raise type(e)(node_def, op, message)
               || tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value prefix/ConvNet/fc_layer_2/bias
               || 	 [[node prefix/ConvNet/fc_layer_2/bias/read (defined at /workspace/graph_utils.py:14) ]]
               ||
               || Caused by op 'prefix/ConvNet/fc_layer_2/bias/read', defined at:
               ||   File "imitation_agent.py", line 136, in <module>
               ||     main()
               ||   File "imitation_agent.py", line 132, in main
               ||     wrap_direct(node=node, protocol=protocol)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/interface.py", line 22, in wrap_direct
               ||     run_loop(node, protocol, args)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 196, in run_loop
               ||     config=config)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 277, in loop
               ||     handle_message_node(parsed, receiver0, context0)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 431, in handle_message_node
               ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/utils.py", line 19, in call_if_fun_exists
               ||     f(**kwargs)
               ||   File "imitation_agent.py", line 71, in on_received_get_commands
               ||     pwm_left, pwm_right = self.compute_action(self.config.current_image)
               ||   File "imitation_agent.py", line 43, in compute_action
               ||     graph = load_graph(frozen_model_filename)
               ||   File "/workspace/graph_utils.py", line 14, in load_graph
               ||     tf.import_graph_def(graph_def, name="prefix")
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
               ||     return func(*args, **kwargs)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/importer.py", line 442, in import_graph_def
               ||     _ProcessNewOps(graph)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/importer.py", line 235, in _ProcessNewOps
               ||     for new_op in graph._add_new_tf_operations(compute_devices=False):  # pylint: disable=protected-access
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3433, in _add_new_tf_operations
               ||     for c_op in c_api_util.new_tf_operations(self)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3433, in <listcomp>
               ||     for c_op in c_api_util.new_tf_operations(self)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3325, in _create_op_from_tf_operation
               ||     ret = Operation(c_op, self)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 1801, in __init__
               ||     self._traceback = tf_stack.extract_stack()
               ||
               || FailedPreconditionError (see above for traceback): Attempting to use uninitialized value prefix/ConvNet/fc_layer_2/bias
               || 	 [[node prefix/ConvNet/fc_layer_2/bias/read (defined at /workspace/graph_utils.py:14) ]]
               ||
               ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/duckietown_challenges/cie_concrete.py", line 675, in scoring_context
    yield cie
  File "experiment_manager.py", line 444, in <module>
    wrap(cie)
  File "experiment_manager.py", line 432, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 136, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 303, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
244064211Bhairav Mehtachallenge-aido_LF-template-pytorchaido3-LFVI-sim-testingstep1-simulationerrornoip-172-31-43-40-104980:04:15
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
243944203Manfred Diazchallenge-aido_LF-template-tensorflowaido3-LFVI-sim-testingstep1-simulationerrornoip-172-31-43-40-104980:04:06
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
243784192Liam Paull 🇨🇦challenge-aido_LF-template-ros - Template solution using ROSaido3-LFVI-sim-validationstep1-simulationhost-errornoip-172-31-43-40-104980:09:22
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 641, in get_cr
    wd, aws_config, copy_to_machine_cache=copy_to_machine_cache
  File "/project/src/duckietown_challenges_runner/runner.py", line 1301, in upload_files
    aws_config, toupload, copy_to_machine_cache=copy_to_machine_cache
  File "/project/src/duckietown_challenges_runner/runner.py", line 1555, in upload
    sha256hex = compute_sha256hex(realfile)
  File "/project/src/duckietown_challenges_runner/runner.py", line 1611, in compute_sha256hex
    res: bytes = subprocess.check_output(cmd)
  File "/usr/lib/python3.6/subprocess.py", line 356, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['shasum', '-a', '256', '/tmp/duckietown/DT18/evaluator/executions/aido3-LFVI-sim-validation/submission4192/step1-simulation-ip-172-31-43-40-10498-job24378/challenge-evaluation-output/episodes/udem1-0-0/camera.mp4']' returned non-zero exit status 1.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
243684179Liam Paull 🇨🇦challenge-aido_LF-baseline-duckietownaido3-LF-sim-validationstep1-simulationerrornoip-172-31-43-40-104980:03:10
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible