Duckietown Challenges Home Challenges Submissions

Evaluator 1429

ID1429
evaluatorp-18382
ownerAndrea Censi 🇨🇭
machineidsc-rudolf
processp-18382
versiond-c:4.0.30;d-c-r:4.0.47;d-s:4.0.22
first heard
last heard
statusinactive
# evaluating
# success9 23659
# timeout
# failed
# error4 23892
# aborted3 23761
# host-error1 23696
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
239843783Andrés Calderón 🇨🇴challenge-aido_LF-template-randomaido2-LF-sim-testing-offseasonstep1-simulationsuccessyesp-183820:13:29
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.5723310224856362
survival_time_median2.549999999999999
deviation-center-line_median0.10099507669212504
in-drivable-lane_median0.4499999999999984


other stats
agent_compute-ego_max0.16326496180366068
agent_compute-ego_mean0.15007449002114306
agent_compute-ego_median0.15122277827202518
agent_compute-ego_min0.14031954765319823
deviation-center-line_max0.2838635700333795
deviation-center-line_mean0.12786165368054708
deviation-center-line_min0.07112529346797256
deviation-heading_max0.5289022616823588
deviation-heading_mean0.3391682685652287
deviation-heading_median0.32928385103533875
deviation-heading_min0.19401296908406307
driven_any_max2.1706243860917107
driven_any_mean0.9759490378276146
driven_any_median1.0117660515775693
driven_any_min0.23626560261560617
driven_lanedir_consec_max1.919400421063832
driven_lanedir_consec_mean0.6896171461819246
driven_lanedir_consec_min0.2264965947033466
driven_lanedir_max1.919400421063832
driven_lanedir_mean0.6896171461819246
driven_lanedir_median0.5723310224856362
driven_lanedir_min0.2264965947033466
in-drivable-lane_max1.7999999999999936
in-drivable-lane_mean0.5899999999999982
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.0243289631429708, "sim_physics": 0.058394273122151695, "survival_time": 2.6999999999999984, "driven_lanedir": 0.5723310224856362, "sim_render-ego": 0.05640847153133816, "in-drivable-lane": 0.9999999999999976, "agent_compute-ego": 0.1463602869598954, "deviation-heading": 0.19401296908406307, "set_robot_commands": 0.0771540844881976, "deviation-center-line": 0.15907928712407893, "driven_lanedir_consec": 0.5723310224856362, "sim_compute_sim_state": 0.03801532586415609, "sim_compute_performance-ego": 0.06202146300563106, "sim_compute_robot_state-ego": 0.06278459672574643}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.9933867744747804, "sim_physics": 0.06738008704839968, "survival_time": 2.549999999999999, "driven_lanedir": 0.8010406009604816, "sim_render-ego": 0.058067550846174654, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.14612258181852453, "deviation-heading": 0.5289022616823588, "set_robot_commands": 0.07825501759847005, "deviation-center-line": 0.13005513204193506, "driven_lanedir_consec": 0.8010406009604816, "sim_compute_sim_state": 0.03892428734723259, "sim_compute_performance-ego": 0.06441568393333286, "sim_compute_robot_state-ego": 0.06583798165414848}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.23626560261560617, "sim_physics": 0.05915343060212977, "survival_time": 0.8500000000000002, "driven_lanedir": 0.2264965947033466, "sim_render-ego": 0.05542822445140166, "in-drivable-lane": 0, "agent_compute-ego": 0.14530173469992244, "deviation-heading": 0.23048898560984332, "set_robot_commands": 0.0768296297858743, "deviation-center-line": 0.0788839760023766, "driven_lanedir_consec": 0.2264965947033466, "sim_compute_sim_state": 0.03933913567486931, "sim_compute_performance-ego": 0.06283043412601247, "sim_compute_robot_state-ego": 0.06437148767359116}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.4346971317320432, "sim_physics": 0.06500534057617187, "survival_time": 1.2500000000000004, "driven_lanedir": 0.40985475638935265, "sim_render-ego": 0.05572074890136719, "in-drivable-lane": 0, "agent_compute-ego": 0.14031954765319823, "deviation-heading": 0.4138862827398061, "set_robot_commands": 0.08002803802490234, "deviation-center-line": 0.08421324241865966, "driven_lanedir_consec": 0.40985475638935265, "sim_compute_sim_state": 0.03839385032653809, "sim_compute_performance-ego": 0.06223978996276856, "sim_compute_robot_state-ego": 0.06569174766540527}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 1.618214457700942, "sim_physics": 0.0581848862804944, "survival_time": 3.949999999999994, "driven_lanedir": 0.7890999076769956, "sim_render-ego": 0.0592879705791232, "in-drivable-lane": 1.7999999999999936, "agent_compute-ego": 0.15122277827202518, "deviation-heading": 0.23606825392187256, "set_robot_commands": 0.07962186125260365, "deviation-center-line": 0.2072381278624381, "driven_lanedir_consec": 0.7890999076769956, "sim_compute_sim_state": 0.038391203819950925, "sim_compute_performance-ego": 0.06415518929686727, "sim_compute_robot_state-ego": 0.06553206564504889}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 1.0038514956833002, "sim_physics": 0.06439094640770737, "survival_time": 2.4499999999999993, "driven_lanedir": 0.7531703123735944, "sim_render-ego": 0.05769737399354273, "in-drivable-lane": 0.4499999999999984, "agent_compute-ego": 0.14173709616369132, "deviation-heading": 0.4596437246381204, "set_robot_commands": 0.07675353361635792, "deviation-center-line": 0.08968283443958727, "driven_lanedir_consec": 0.7531703123735944, "sim_compute_sim_state": 0.038428082758066605, "sim_compute_performance-ego": 0.0627185617174421, "sim_compute_robot_state-ego": 0.06584782503089126}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 1.0542898001504375, "sim_physics": 0.05648315378597805, "survival_time": 2.799999999999998, "driven_lanedir": 0.5389398655375865, "sim_render-ego": 0.05778030838285174, "in-drivable-lane": 1.1499999999999972, "agent_compute-ego": 0.15178427951676504, "deviation-heading": 0.4109275195546752, "set_robot_commands": 0.07789337635040283, "deviation-center-line": 0.10786249131209884, "driven_lanedir_consec": 0.5389398655375865, "sim_compute_sim_state": 0.03809154885155814, "sim_compute_performance-ego": 0.06406048366001674, "sim_compute_robot_state-ego": 0.06591096946171351}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 0.3164475681403097, "sim_physics": 0.0782107739221482, "survival_time": 1.0500000000000005, "driven_lanedir": 0.3093025166124813, "sim_render-ego": 0.06221285320463635, "in-drivable-lane": 0, "agent_compute-ego": 0.15147160348438082, "deviation-heading": 0.2242052814634115, "set_robot_commands": 0.08071917579287574, "deviation-center-line": 0.09758615464743424, "driven_lanedir_consec": 0.3093025166124813, "sim_compute_sim_state": 0.03838107699439639, "sim_compute_performance-ego": 0.0662908327011835, "sim_compute_robot_state-ego": 0.06848633857000441}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 1.2018528489510765, "sim_physics": 0.07768040188288285, "survival_time": 2.9499999999999975, "driven_lanedir": 0.6922532395203, "sim_render-ego": 0.059112156851816984, "in-drivable-lane": 1.0499999999999965, "agent_compute-ego": 0.1407552896919897, "deviation-heading": 0.32928385103533875, "set_robot_commands": 0.08409637111728474, "deviation-center-line": 0.07112529346797256, "driven_lanedir_consec": 0.6922532395203, "sim_compute_sim_state": 0.03887850955381232, "sim_compute_performance-ego": 0.0643187054133011, "sim_compute_robot_state-ego": 0.0690318002539166}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 1.0435481553257333, "sim_physics": 0.05698984319513494, "survival_time": 2.7499999999999982, "driven_lanedir": 0.5204405548974873, "sim_render-ego": 0.05902069698680531, "in-drivable-lane": 1.1499999999999977, "agent_compute-ego": 0.15081751996820622, "deviation-heading": 0.33296447766287096, "set_robot_commands": 0.07700044892051003, "deviation-center-line": 0.0950267943453044, "driven_lanedir_consec": 0.5204405548974873, "sim_compute_sim_state": 0.03861750255931507, "sim_compute_performance-ego": 0.06256695660677823, "sim_compute_robot_state-ego": 0.06331863836808638}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 1.5954548672193265, "sim_physics": 0.06661432129996163, "survival_time": 3.8499999999999943, "driven_lanedir": 1.1042463557848754, "sim_render-ego": 0.059922555824378866, "in-drivable-lane": 1.0499999999999965, "agent_compute-ego": 0.1540932036065436, "deviation-heading": 0.3566810710676767, "set_robot_commands": 0.08138021865448394, "deviation-center-line": 0.22513914436082189, "driven_lanedir_consec": 1.1042463557848754, "sim_compute_sim_state": 0.04112004614495612, "sim_compute_performance-ego": 0.06477107010878526, "sim_compute_robot_state-ego": 0.06823841627542075}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 0.49030923107489305, "sim_physics": 0.06380014760153634, "survival_time": 1.4000000000000006, "driven_lanedir": 0.4826393189617785, "sim_render-ego": 0.06080632550375802, "in-drivable-lane": 0, "agent_compute-ego": 0.15464087894984654, "deviation-heading": 0.23322336656472228, "set_robot_commands": 0.07914922918592181, "deviation-center-line": 0.10099507669212504, "driven_lanedir_consec": 0.4826393189617785, "sim_compute_sim_state": 0.040232837200164795, "sim_compute_performance-ego": 0.0663967388016837, "sim_compute_robot_state-ego": 0.06742008243288312}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 1.0117660515775693, "sim_physics": 0.06366287960725672, "survival_time": 2.549999999999999, "driven_lanedir": 0.791717039833255, "sim_render-ego": 0.06215403126735313, "in-drivable-lane": 0.3999999999999986, "agent_compute-ego": 0.16326496180366068, "deviation-heading": 0.320449356593834, "set_robot_commands": 0.07975851320752911, "deviation-center-line": 0.1017166571540705, "driven_lanedir_consec": 0.791717039833255, "sim_compute_sim_state": 0.04215327898661295, "sim_compute_performance-ego": 0.06714172924266142, "sim_compute_robot_state-ego": 0.0687425931294759}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 0.44419823353351984, "sim_physics": 0.06446079854611997, "survival_time": 1.3500000000000003, "driven_lanedir": 0.4333246859278659, "sim_render-ego": 0.05966733120105885, "in-drivable-lane": 0, "agent_compute-ego": 0.15627073358606408, "deviation-heading": 0.2988926743481924, "set_robot_commands": 0.07984165792111997, "deviation-center-line": 0.08545702330592322, "driven_lanedir_consec": 0.4333246859278659, "sim_compute_sim_state": 0.04216157065497504, "sim_compute_performance-ego": 0.0645909485993562, "sim_compute_robot_state-ego": 0.06763809698599356}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 2.1706243860917107, "sim_physics": 0.05866357391955806, "survival_time": 5.09999999999999, "driven_lanedir": 1.919400421063832, "sim_render-ego": 0.05935200756671382, "in-drivable-lane": 0.4999999999999982, "agent_compute-ego": 0.15695485414243213, "deviation-heading": 0.5178939525116442, "set_robot_commands": 0.07874716964422487, "deviation-center-line": 0.2838635700333795, "driven_lanedir_consec": 1.919400421063832, "sim_compute_sim_state": 0.041365892279381845, "sim_compute_performance-ego": 0.06554939232620538, "sim_compute_robot_state-ego": 0.06734466085246965}}
set_robot_commands_max0.08409637111728474
set_robot_commands_mean0.07914855503738392
set_robot_commands_median0.07914922918592181
set_robot_commands_min0.07675353361635792
sim_compute_performance-ego_max0.06714172924266142
sim_compute_performance-ego_mean0.06427119863346839
sim_compute_performance-ego_median0.0643187054133011
sim_compute_performance-ego_min0.06202146300563106
sim_compute_robot_state-ego_max0.0690318002539166
sim_compute_robot_state-ego_mean0.06641315338165302
sim_compute_robot_state-ego_median0.06591096946171351
sim_compute_robot_state-ego_min0.06278459672574643
sim_compute_sim_state_max0.04216157065497504
sim_compute_sim_state_mean0.03949960993439909
sim_compute_sim_state_median0.03887850955381232
sim_compute_sim_state_min0.03801532586415609
sim_physics_max0.0782107739221482
sim_physics_mean0.06393832385317541
sim_physics_median0.06380014760153634
sim_physics_min0.05648315378597805
sim_render-ego_max0.06221285320463635
sim_render-ego_mean0.0588425738061547
sim_render-ego_median0.059112156851816984
sim_render-ego_min0.05542822445140166
simulation-passed1
survival_time_max5.09999999999999
survival_time_mean2.503333333333331
survival_time_min0.8500000000000002
No reset possible
239833782Yeong-Jia ChuBaseline solution using reinforcement learningaido2-LF-sim-validation-offseasonstep1-simulationerroryesp-183820:00:41
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
239743715jiang pengfinal test - whaido2-LF-real-validationeval0-videos-autobotserroryesp-183823:01:03
Waited 10802 seconds [...]
Waited 10802 seconds for container to finish. Giving up. 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
239463748sebastian glatzaido2-LFV-real-validationeval0-videos-autobotserroryesp-183823:00:57
Waited 10801 seconds [...]
Waited 10801 seconds for container to finish. Giving up. 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
238923776Anastasiya Nikolskaya 🇷🇺Offset-based Oneaido2-LFVI-real-validationeval0-videos-autobotserroryesp-183823:00:54
Waited 10800 seconds [...]
Waited 10800 seconds for container to finish. Giving up. 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
238373739jiang pengLF sim test - whaido2-LF-real-validationeval0-videos-watchtowerssuccessyesp-183821:22:18
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
238363739jiang pengLF sim test - whaido2-LF-real-validationeval0-videos-autobotssuccessyesp-183820:02:35
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
238173777Tsz Fung Yip 🇭🇰aido2-LF-real-validationeval0-videos-watchtowerssuccessyesp-183821:24:02
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
238133777Tsz Fung Yip 🇭🇰aido2-LF-real-validationeval0-videos-autobotssuccessyesp-183820:03:19
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
238043758Julian ZillyBaseline-IL-logs-tensorflowaido2-LFV-real-validationeval0-videos-autobotssuccessyesp-183820:10:41
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
237883768Anastasiya Nikolskaya 🇷🇺Boxed Solution 0.3aido2-LF-real-validationeval0-videos-autobotsabortedyesp-183821:17:04
KeyboardInterrupt: T [...]
KeyboardInterrupt:
Traceback (most recent call last):
  File "/home/andrea/dt-env-developer/src/duckietown-challenges-runner/src/duckietown_challenges_runner/runner.py", line 820, in run_one
    time.sleep(5)
KeyboardInterrupt
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
237823761Julian ZillyBaseline-IL-logs-tensorflowaido2-LFVI-real-validationeval0-videos-autobotsabortedyesp-183821:06:00
KeyboardInterrupt: T [...]
KeyboardInterrupt:
Traceback (most recent call last):
  File "/home/andrea/dt-env-developer/src/duckietown-challenges-runner/src/duckietown_challenges_runner/runner.py", line 820, in run_one
    time.sleep(5)
KeyboardInterrupt
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
237693608Riza Velioglu 🇩🇪aido2-LFV-real-validationeval1-videos-watchtowerssuccessyesp-183821:36:40
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
237613741jiang pengLF sim test - whaido2-LFV-real-validationeval0-videos-autobotsabortedyesp-183821:22:32
KeyboardInterrupt: T [...]
KeyboardInterrupt:
Traceback (most recent call last):
  File "/home/andrea/dt-env-developer/src/duckietown-challenges-runner/src/duckietown_challenges_runner/runner.py", line 820, in run_one
    time.sleep(5)
KeyboardInterrupt
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
236963631Nicky Eichmannaido2-LFVI-real-validationeval0-videos-watchtowershost-erroryesp-183822:58:21
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/home/andrea/dt-env-developer/src/duckietown-challenges-runner/src/duckietown_challenges_runner/runner.py", line 544, in get_cr
    uploaded = upload_files(wd, aws_config, copy_to_machine_cache=copy_to_machine_cache)
  File "/home/andrea/dt-env-developer/src/duckietown-challenges-runner/src/duckietown_challenges_runner/runner.py", line 1123, in upload_files
    uploaded = upload(aws_config, toupload, copy_to_machine_cache=copy_to_machine_cache)
  File "/home/andrea/dt-env-developer/src/duckietown-challenges-runner/src/duckietown_challenges_runner/runner.py", line 1363, in upload
    copy_to_cache(realfile, sha256hex)
  File "/home/andrea/dt-env-developer/src/duckietown-challenges-runner/src/duckietown_challenges_runner/runner_cache.py", line 41, in copy_to_cache
    shutil.copy(fn, have)
  File "/home/andrea/.local/share/virtualenvs/dt-env-developer-q48V9oeQ/lib/python3.7/shutil.py", line 245, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/home/andrea/.local/share/virtualenvs/dt-env-developer-q48V9oeQ/lib/python3.7/shutil.py", line 122, in copyfile
    copyfileobj(fsrc, fdst)
OSError: [Errno 28] No space left on device
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
236613593Victor Adolfo Romero Cano 🇨🇴aido2-LF-real-validationeval1-videos-watchtowerssuccessyesp-183820:56:00
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
236593593Victor Adolfo Romero Cano 🇨🇴aido2-LF-real-validationeval1-videos-autobotssuccessyesp-183820:04:30
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible