Duckietown Challenges Home Challenges Submissions

Evaluator 1055

ID1055
evaluatornutonomy-P50-17188
ownerAndrea Censi 🇨🇭
machinenutonomy-P50
processnutonomy-P50-17188
versiond-c:4.0.28;d-c-r:4.0.39;d-s:4.0.21
first heard
last heard
statusinactive
# evaluating
# success2 21988
# timeout
# failed
# error
# aborted1 21974
# host-error1 21977
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
219933377Angel Woo 🇭🇰Baseline-IL-logs-tensorflowaido2-LF-sim-validationstep1-simulationsuccessnonutonomy-P50-171880:06:24
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.359945565079975
survival_time_median3.7499999999999942
deviation-center-line_median0.23864615819168675
in-drivable-lane_median0.09999999999999964


other stats
agent_compute-ego_max0.1337392560640971
agent_compute-ego_mean0.11070012300117764
agent_compute-ego_median0.10519737561543782
agent_compute-ego_min0.10372373792860244
deviation-center-line_max0.520789033534116
deviation-center-line_mean0.2717071769124991
deviation-center-line_min0.17648732541296272
deviation-heading_max1.5042111958808144
deviation-heading_mean1.2052519830265396
deviation-heading_median1.1486841087580049
deviation-heading_min0.9318630729637396
driven_any_max1.2293388836512025
driven_any_mean0.5856850751963685
driven_any_median0.40568365509489707
driven_any_min0.19431740742645395
driven_lanedir_consec_max1.2193750465477722
driven_lanedir_consec_mean0.5245571313965561
driven_lanedir_consec_min0.1630332041872764
driven_lanedir_max1.2193750465477722
driven_lanedir_mean0.5245571313965561
driven_lanedir_median0.359945565079975
driven_lanedir_min0.1630332041872764
in-drivable-lane_max1.5500000000000087
in-drivable-lane_mean0.3700000000000015
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.2293388836512025, "sim_physics": 0.0394685435295105, "survival_time": 14.950000000000076, "driven_lanedir": 1.2193750465477722, "sim_render-ego": 0.0325927209854126, "in-drivable-lane": 0, "agent_compute-ego": 0.1337392560640971, "deviation-heading": 1.5042111958808144, "set_robot_commands": 0.05049805164337158, "deviation-center-line": 0.520789033534116, "driven_lanedir_consec": 1.2193750465477722, "sim_compute_sim_state": 0.020530121326446535, "sim_compute_performance-ego": 0.03560329675674438, "sim_compute_robot_state-ego": 0.03741559425989787}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.8374869926376158, "sim_physics": 0.037744912724036954, "survival_time": 8.84999999999999, "driven_lanedir": 0.6764706165315955, "sim_render-ego": 0.03125782605618407, "in-drivable-lane": 1.5500000000000087, "agent_compute-ego": 0.105814062269394, "deviation-heading": 1.1486841087580049, "set_robot_commands": 0.04870454335616807, "deviation-center-line": 0.23864615819168675, "driven_lanedir_consec": 0.6764706165315955, "sim_compute_sim_state": 0.01978607770413329, "sim_compute_performance-ego": 0.03463686522790941, "sim_compute_robot_state-ego": 0.036274142184499966}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.19431740742645395, "sim_physics": 0.03891452683342827, "survival_time": 2.25, "driven_lanedir": 0.1630332041872764, "sim_render-ego": 0.03097281455993652, "in-drivable-lane": 0.14999999999999947, "agent_compute-ego": 0.10372373792860244, "deviation-heading": 0.9318630729637396, "set_robot_commands": 0.049203141530354814, "deviation-center-line": 0.2434894883029827, "driven_lanedir_consec": 0.1630332041872764, "sim_compute_sim_state": 0.019905699623955626, "sim_compute_performance-ego": 0.034728044933742944, "sim_compute_robot_state-ego": 0.03943431642320421}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.2615984371716733, "sim_physics": 0.0416698694229126, "survival_time": 2.499999999999999, "driven_lanedir": 0.2039612246361613, "sim_render-ego": 0.03190654754638672, "in-drivable-lane": 0.04999999999999982, "agent_compute-ego": 0.10502618312835692, "deviation-heading": 1.41683802682294, "set_robot_commands": 0.0492133903503418, "deviation-center-line": 0.17648732541296272, "driven_lanedir_consec": 0.2039612246361613, "sim_compute_sim_state": 0.019844350814819337, "sim_compute_performance-ego": 0.03432457447052002, "sim_compute_robot_state-ego": 0.03632218360900879}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.40568365509489707, "sim_physics": 0.0400682004292806, "survival_time": 3.7499999999999942, "driven_lanedir": 0.359945565079975, "sim_render-ego": 0.03203426996866862, "in-drivable-lane": 0.09999999999999964, "agent_compute-ego": 0.10519737561543782, "deviation-heading": 1.0246635107071995, "set_robot_commands": 0.04892118771870931, "deviation-center-line": 0.17912387912074734, "driven_lanedir_consec": 0.359945565079975, "sim_compute_sim_state": 0.019895267486572263, "sim_compute_performance-ego": 0.03444735209147135, "sim_compute_robot_state-ego": 0.03664422035217285}}
set_robot_commands_max0.05049805164337158
set_robot_commands_mean0.04930806291978912
set_robot_commands_median0.049203141530354814
set_robot_commands_min0.04870454335616807
sim_compute_performance-ego_max0.03560329675674438
sim_compute_performance-ego_mean0.03474802669607762
sim_compute_performance-ego_median0.03463686522790941
sim_compute_performance-ego_min0.03432457447052002
sim_compute_robot_state-ego_max0.03943431642320421
sim_compute_robot_state-ego_mean0.03721809136575674
sim_compute_robot_state-ego_median0.03664422035217285
sim_compute_robot_state-ego_min0.036274142184499966
sim_compute_sim_state_max0.020530121326446535
sim_compute_sim_state_mean0.01999230339118541
sim_compute_sim_state_median0.019895267486572263
sim_compute_sim_state_min0.01978607770413329
sim_physics_max0.0416698694229126
sim_physics_mean0.039573210587833786
sim_physics_median0.0394685435295105
sim_physics_min0.037744912724036954
sim_render-ego_max0.0325927209854126
sim_render-ego_mean0.031752835823317706
sim_render-ego_median0.03190654754638672
sim_render-ego_min0.03097281455993652
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean6.460000000000012
survival_time_min2.25
No reset possible
219883376Lei Chenchallenge-aido_LF-baseline-duckietownaido2-LF-sim-validationstep1-simulationsuccessnonutonomy-P50-171880:07:37
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.0999974081549406
survival_time_median7.94999999999998
deviation-center-line_median0.4636024032578443
in-drivable-lane_median0.30000000000000426


other stats
agent_compute-ego_max0.05386954247951507
agent_compute-ego_mean0.053061551476588975
agent_compute-ego_median0.05292630737478083
agent_compute-ego_min0.05253244945838016
deviation-center-line_max0.8806668824903146
deviation-center-line_mean0.5510181937862011
deviation-center-line_min0.3453832142297992
deviation-heading_max5.0615127742139325
deviation-heading_mean2.60676286913447
deviation-heading_median2.481851441345405
deviation-heading_min0.5709311703588489
driven_any_max2.3411533017430974
driven_any_mean1.2288482972275658
driven_any_median1.2240044676966393
driven_any_min0.5892284233478767
driven_lanedir_consec_max1.2179768128330544
driven_lanedir_consec_mean0.8496682456025259
driven_lanedir_consec_min0.21070249865609705
driven_lanedir_max1.3629275037955328
driven_lanedir_mean0.9244154187407828
driven_lanedir_median1.0999974081549406
driven_lanedir_min0.3587294229445502
in-drivable-lane_max4.200000000000001
in-drivable-lane_mean1.030000000000001
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 2.3411533017430974, "sim_physics": 0.03991844415664673, "survival_time": 14.950000000000076, "driven_lanedir": 1.3629275037955328, "sim_render-ego": 0.03389791488647461, "in-drivable-lane": 4.200000000000001, "agent_compute-ego": 0.05291959683100383, "deviation-heading": 5.0615127742139325, "set_robot_commands": 0.052494850953420005, "deviation-center-line": 0.8806668824903146, "driven_lanedir_consec": 1.1372185623926994, "sim_compute_sim_state": 0.021322381496429444, "sim_compute_performance-ego": 0.03706813891728719, "sim_compute_robot_state-ego": 0.03827229102452596}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.6292597443570077, "sim_physics": 0.04019383823170381, "survival_time": 4.249999999999993, "driven_lanedir": 0.3587294229445502, "sim_render-ego": 0.03381828700794893, "in-drivable-lane": 0.6499999999999997, "agent_compute-ego": 0.053059861239265, "deviation-heading": 2.305667469272767, "set_robot_commands": 0.05152304873746984, "deviation-center-line": 0.3453832142297992, "driven_lanedir_consec": 0.21070249865609705, "sim_compute_sim_state": 0.021500811857335708, "sim_compute_performance-ego": 0.03697052843430463, "sim_compute_robot_state-ego": 0.03794831668629366}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.5892284233478767, "sim_physics": 0.03912104666233063, "survival_time": 3.999999999999994, "driven_lanedir": 0.5824459459758375, "sim_render-ego": 0.03362565040588379, "in-drivable-lane": 0, "agent_compute-ego": 0.05386954247951507, "deviation-heading": 0.5709311703588489, "set_robot_commands": 0.052143269777297975, "deviation-center-line": 0.4285700982939006, "driven_lanedir_consec": 0.5824459459758375, "sim_compute_sim_state": 0.02119922637939453, "sim_compute_performance-ego": 0.037234410643577576, "sim_compute_robot_state-ego": 0.0379274308681488}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 1.3605955489932082, "sim_physics": 0.04089944606477564, "survival_time": 8.79999999999999, "driven_lanedir": 1.2179768128330544, "sim_render-ego": 0.03355961496179754, "in-drivable-lane": 0.30000000000000426, "agent_compute-ego": 0.05292630737478083, "deviation-heading": 2.481851441345405, "set_robot_commands": 0.052272334694862366, "deviation-center-line": 0.6368683706591467, "driven_lanedir_consec": 1.2179768128330544, "sim_compute_sim_state": 0.021132474595850163, "sim_compute_performance-ego": 0.03683774308724837, "sim_compute_robot_state-ego": 0.03791849586096677}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 1.2240044676966393, "sim_physics": 0.039421823789488594, "survival_time": 7.94999999999998, "driven_lanedir": 1.0999974081549406, "sim_render-ego": 0.033671401581674254, "in-drivable-lane": 0, "agent_compute-ego": 0.05253244945838016, "deviation-heading": 2.6138514904813963, "set_robot_commands": 0.05126056431224511, "deviation-center-line": 0.4636024032578443, "driven_lanedir_consec": 1.0999974081549406, "sim_compute_sim_state": 0.020968750587799265, "sim_compute_performance-ego": 0.03673989068037309, "sim_compute_robot_state-ego": 0.03856913998441877}}
set_robot_commands_max0.052494850953420005
set_robot_commands_mean0.05193881369505906
set_robot_commands_median0.052143269777297975
set_robot_commands_min0.05126056431224511
sim_compute_performance-ego_max0.037234410643577576
sim_compute_performance-ego_mean0.03697014235255817
sim_compute_performance-ego_median0.03697052843430463
sim_compute_performance-ego_min0.03673989068037309
sim_compute_robot_state-ego_max0.03856913998441877
sim_compute_robot_state-ego_mean0.03812713488487079
sim_compute_robot_state-ego_median0.03794831668629366
sim_compute_robot_state-ego_min0.03791849586096677
sim_compute_sim_state_max0.021500811857335708
sim_compute_sim_state_mean0.02122472898336182
sim_compute_sim_state_median0.02119922637939453
sim_compute_sim_state_min0.020968750587799265
sim_physics_max0.04089944606477564
sim_physics_mean0.03991091978098908
sim_physics_median0.03991844415664673
sim_physics_min0.03912104666233063
sim_render-ego_max0.03389791488647461
sim_render-ego_mean0.03371457376875582
sim_render-ego_median0.033671401581674254
sim_render-ego_min0.03355961496179754
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean7.990000000000007
survival_time_min3.999999999999994
No reset possible
219773368Amaury Camus 🇨🇭challenge-aido_LF-template-randomaido2-LFV-sim-validationstep1-simulationhost-errornonutonomy-P50-171880:00:37
Error while running [...]

stderr | Pulling solution       ... 
stderr | Pulling scenario_maker ... 
stderr | Pulling evaluator      ... done
stderr | 
stderr | ERROR: for simulator  error pulling image configuration: Get https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blobs/sha256/3e/3ecc0062b0ecc477fa7f260c0ab6d4bd752315fbabf8a5895e0aecd90fb7a4a3/data?verify=1557769588-AYZIrW%2B5WrMR2loeW93UjmlO%2BPc%3D: net/http: TLS handshake timeout
stderr | error pulling image configuration: Get https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blobs/sha256/3e/3ecc0062b0ecc477fa7f260c0ab6d4bd752315fbabf8a5895e0aecd90fb7a4a3/data?verify=1557769588-AYZIrW%2B5WrMR2loeW93UjmlO%2BPc%3D: net/http: TLS handshake timeout
stderr |
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
219743375Andrea Censi 🇨🇭Solution templateaido2-MS-ipfsstep2abortednonutonomy-P50-171880:00:23
Unexpected exception [...]
Unexpected exception:
Traceback (most recent call last):
  File "/project/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 624, in scoring_context
    yield cie
  File "eval_ipfs.py", line 87, in <module>
    score(cie)
  File "eval_ipfs.py", line 77, in score
    score_step2(cie)
  File "eval_ipfs.py", line 59, in score_step2
    fn = cie.get_completed_step_evaluation_file(STEP1, os.path.join(basename, 'one.txt'))
  File "/project/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 429, in get_completed_step_evaluation_file
    return get_completed_step_evaluation_file(self.root, step_name, basename)
  File "/project/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 467, in get_completed_step_evaluation_file
    raise KeyError(msg)
KeyError: "Cannot find /challenges/previous-steps/step1/challenge-evaluation-output/mylogs/one.txt; know ['regular.txt']"
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible