Duckietown Challenges Home Challenges Submissions

Evaluator 488

ID488
evaluatornutonomy-P50-5849
ownerAndrea Censi 🇨🇭
machinenutonomy-P50
processnutonomy-P50-5849
versionDC:4.0.4;DCR:4.0.5;DTS:4.0.6
first heard
last heard
statusinactive
# evaluating
# success27 16899
# timeout
# failed
# error1 16961
# aborted
# host-error1 16955
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
169652253Julian ZillyBaseline solution using imitation learning from logsaido2_LF_r2-v4step1-simulationsuccessnonutonomy-P50-58490:10:05
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.7093853387119008
deviation-center-line_median0.049951296106660166
in-drivable-lane_median0.7249999999999974


other stats
deviation-center-line_max0.0655443072471126
deviation-center-line_mean0.051310129973861786
deviation-center-line_min0.0425382635831745
deviation-heading_max0.3443962673473085
deviation-heading_mean0.32623740442907956
deviation-heading_median0.33549133680865645
deviation-heading_min0.29887715619413224
driven_any_max2.1602376773534653
driven_any_mean1.186889319168355
driven_any_median0.9923541772422052
driven_any_min0.9918955785816944
driven_lanedir_consec_max1.9434353684594352
driven_lanedir_consec_mean0.9063883551865018
driven_lanedir_consec_min0.6573799722057614
driven_lanedir_max1.9434353684594352
driven_lanedir_mean0.9063883551865018
driven_lanedir_median0.7093853387119008
driven_lanedir_min0.6573799722057614
in-drivable-lane_max0.849999999999997
in-drivable-lane_mean0.7166666666666641
in-drivable-lane_min0.549999999999998
per-episodes
details{"4way-0-0": {"driven_any": 2.1602376773534653, "driven_lanedir": 1.9434353684594352, "in-drivable-lane": 0.549999999999998, "deviation-heading": 0.3267037102009071, "deviation-center-line": 0.0655443072471126, "driven_lanedir_consec": 1.9434353684594352}, "4way-1-0": {"driven_any": 0.992293326300768, "driven_lanedir": 0.6574255540192131, "in-drivable-lane": 0.849999999999997, "deviation-heading": 0.3443962673473085, "deviation-center-line": 0.04256558567737999, "driven_lanedir_consec": 0.6574255540192131}, "udem1-0-0": {"driven_any": 0.9918955785816944, "driven_lanedir": 0.6573799722057614, "in-drivable-lane": 0.849999999999997, "deviation-heading": 0.3442789634164058, "deviation-center-line": 0.0425382635831745, "driven_lanedir_consec": 0.6573799722057614}, "udem1-1-0": {"driven_any": 0.9918955785816944, "driven_lanedir": 0.6573799722057614, "in-drivable-lane": 0.849999999999997, "deviation-heading": 0.3442789634164058, "deviation-center-line": 0.0425382635831745, "driven_lanedir_consec": 0.6573799722057614}, "small_loop-0-0": {"driven_any": 0.992598726008866, "driven_lanedir": 0.7613641408242504, "in-drivable-lane": 0.5999999999999979, "deviation-heading": 0.29887715619413224, "deviation-center-line": 0.05733735321638882, "driven_lanedir_consec": 0.7613641408242504}, "small_loop-1-0": {"driven_any": 0.9924150281836428, "driven_lanedir": 0.7613451234045885, "in-drivable-lane": 0.5999999999999979, "deviation-heading": 0.29888936599931776, "deviation-center-line": 0.057337006535940344, "driven_lanedir_consec": 0.7613451234045885}}
simulation-passed1
No reset possible
169612259Bhairav MehtaPyTorch templateaido2_LF_r2-v4step1-simulationerrornonutonomy-P50-58490:00:56
The result file is n [...]
The result file is not found. This usually means that the evaluator did not finish
and some times that there was an import error.

Check the evaluator log to see what happened.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
169552264Bhairav MehtaPyTorch templateaido2_LF_r2-v4step1-simulationhost-errornonutonomy-P50-58490:02:40
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 418, in go_
    challenge_step_name, project, do_pull, debug_volumes=debug_volumes)
  File "/project/src/duckietown_challenges_runner/runner.py", line 521, in run_single
    write_logs(wd, project, services=config['services'])
  File "/project/src/duckietown_challenges_runner/runner.py", line 861, in write_logs
    f.write(logs)
UnicodeEncodeError: 'ascii' codec can't encode character '\u2502' in position 5939: ordinal not in range(128)
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
169532266Andrea Censi 🇨🇭PredictorRandomaido2-PREDstep1-simulationsuccessnonutonomy-P50-58490:00:35
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L10.317355069758272
error_L20.14406007588602807


No reset possible
169502270Andrea Censi 🇨🇭PredictorLastaido2-PREDstep1-simulationsuccessnonutonomy-P50-58490:00:33
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L10.11087317870517235
error_L20.020110285597291652


No reset possible
169472273Andrea Censi 🇨🇭PredictorRandomaido2-PREDstep1-simulationsuccessnonutonomy-P50-58490:00:33
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L10.3126644283696903
error_L20.14625357788145582


No reset possible
169442275Andrea Censi 🇨🇭Solution templateaido2_LFV_r1-v4step1-simulationsuccessnonutonomy-P50-58490:13:02
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.9306017070461028
deviation-center-line_median0.2467154410917354
in-drivable-lane_median2.299999999999992


other stats
deviation-center-line_max0.34338570913942046
deviation-center-line_mean0.25030164980112507
deviation-center-line_min0.16439000788160887
deviation-heading_max1.6095489732413446
deviation-heading_mean0.97710984480811
deviation-heading_median0.9609461645189982
deviation-heading_min0.3769980769530998
driven_any_max2.234179003106766
driven_any_mean1.8425421206705448
driven_any_median1.873567146510747
driven_any_min1.3888551865539178
driven_lanedir_consec_max1.3390397100187108
driven_lanedir_consec_mean0.983516144317161
driven_lanedir_consec_min0.7338214531577274
driven_lanedir_max1.7355909832406682
driven_lanedir_mean1.089808351204569
driven_lanedir_median0.944910484209941
driven_lanedir_min0.7338214531577274
in-drivable-lane_max3.149999999999989
in-drivable-lane_mean2.224999999999992
in-drivable-lane_min1.149999999999996
per-episodes
details{"4way-0-0": {"driven_any": 2.1748492745499397, "driven_lanedir": 1.0586363759652746, "in-drivable-lane": 3.149999999999989, "deviation-heading": 1.6095489732413446, "deviation-center-line": 0.3286137010320844, "driven_lanedir_consec": 1.030018821637598}, "4way-1-0": {"driven_any": 2.234179003106766, "driven_lanedir": 1.7355909832406682, "in-drivable-lane": 1.149999999999996, "deviation-heading": 1.530260565460663, "deviation-center-line": 0.34338570913942046, "driven_lanedir_consec": 1.3390397100187108}, "udem1-0-0": {"driven_any": 1.3888551865539178, "driven_lanedir": 0.8311845924546073, "in-drivable-lane": 1.7999999999999936, "deviation-heading": 0.39163176357733337, "deviation-center-line": 0.16481718115138638, "driven_lanedir_consec": 0.8311845924546073}, "udem1-1-0": {"driven_any": 1.5722850184715549, "driven_lanedir": 0.7338214531577274, "in-drivable-lane": 2.79999999999999, "deviation-heading": 0.3769980769530998, "deviation-center-line": 0.16439000788160887, "driven_lanedir_consec": 0.7338214531577274}}
simulation-passed1
No reset possible
169432248Andrea Censi 🇨🇭Solution templateaido1_luck-v4step1successnonutonomy-P50-58490:00:27
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score194.4171955104927


No reset possible
169402248Andrea Censi 🇨🇭Solution templateaido1_luck-v4step1successnonutonomy-P50-58490:00:30
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score168.03832825965918


No reset possible
169392254Andrea Censi 🇨🇭Solution templateaido1_luck-v4step1successnonutonomy-P50-58490:00:25
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score138.0933898969576


No reset possible
169372254Andrea Censi 🇨🇭Solution templateaido1_luck-v4step1successnonutonomy-P50-58490:00:27
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score112.359247527821964


No reset possible
169352248Andrea Censi 🇨🇭Solution templateaido1_luck-v4step1successnonutonomy-P50-58490:00:27
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score159.387118039221775


No reset possible
169332248Andrea Censi 🇨🇭Solution templateaido1_luck-v4step1successnonutonomy-P50-58490:00:26
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score168.37348653447845


No reset possible
169302248Andrea Censi 🇨🇭Solution templateaido1_luck-v4step1successnonutonomy-P50-58490:00:26
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score136.46591537111212


No reset possible
169292248Andrea Censi 🇨🇭Solution templateaido1_luck-v4step1successnonutonomy-P50-58490:00:26
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score149.78332984287719


No reset possible
169272254Andrea Censi 🇨🇭Solution templateaido1_luck-v4step1successnonutonomy-P50-58490:00:29
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score193.801923035422


No reset possible
169242254Andrea Censi 🇨🇭Solution templateaido1_luck-v4step1successnonutonomy-P50-58490:00:30
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score167.72614307154655


No reset possible
169222254Andrea Censi 🇨🇭Solution templateaido1_luck-v4step1successnonutonomy-P50-58490:00:26
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score116.921291938775653


No reset possible
169202248Andrea Censi 🇨🇭Solution templateaido1_luck-v4step1successnonutonomy-P50-58490:00:27
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score177.26052415044764


No reset possible
169172248Andrea Censi 🇨🇭Solution templateaido1_luck-v4step1successnonutonomy-P50-58490:00:31
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score170.45453317870565


No reset possible
169152254Andrea Censi 🇨🇭Solution templateaido1_luck-v4step1successnonutonomy-P50-58490:00:24
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score172.44331849976571


No reset possible
169132254Andrea Censi 🇨🇭Solution templateaido1_luck-v4step1successnonutonomy-P50-58490:00:28
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score117.4176721434294


No reset possible
169102248Andrea Censi 🇨🇭Solution templateaido1_luck-v4step1successnonutonomy-P50-58490:00:31
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score190.80028920543523


No reset possible
169092254Andrea Censi 🇨🇭Solution templateaido1_luck-v4step1successnonutonomy-P50-58490:00:26
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score170.48285553376232


No reset possible
169062254Andrea Censi 🇨🇭Solution templateaido1_luck-v4step1successnonutonomy-P50-58490:00:26
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score16.931568063709932


No reset possible
169052254Andrea Censi 🇨🇭Solution templateaido1_luck-v4step1successnonutonomy-P50-58490:00:22
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score179.93197720291141


No reset possible
169032254Andrea Censi 🇨🇭Solution templateaido1_luck-v4step1successnonutonomy-P50-58490:00:30
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score148.33349568581161


No reset possible
169002248Andrea Censi 🇨🇭Solution templateaido1_luck-v4step1successnonutonomy-P50-58490:00:28
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score124.830342341798115


No reset possible
168992248Andrea Censi 🇨🇭Solution templateaido1_luck-v4step1successnonutonomy-P50-58490:00:30
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score19.983877838369551


No reset possible