Duckietown Challenges Home Challenges Submissions

Submission 2907

Submission2907
Competingyes
Challengeaido2-LFVI-sim-testing
UserJulian Zilly
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobsstep1-simulation: 22789
Next
User labelBaseline-IL-logs-tensorflow
Admin priority50
Blessingn/a
User priority50

The highlights are available only to the owner and the admins.

Evaluation jobs for this submission

See previous jobs for previous versions of challenges
Job IDstepstatusup to datedate starteddate completeddurationmessage
22789step1-simulationsuccessyes0:31:20
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.466627654289285
survival_time_median6.099999999999986
deviation-center-line_median0.33341663981606123
in-drivable-lane_median0


other stats
agent_compute-ego_max0.1145672897497813
agent_compute-ego_mean0.11280293562696216
agent_compute-ego_median0.11285812714520622
agent_compute-ego_min0.1089727467504041
deviation-center-line_max1.0678270161928525
deviation-center-line_mean0.3924833225550088
deviation-center-line_min0.16662188238079648
deviation-heading_max3.00376928154286
deviation-heading_mean1.182722648012555
deviation-heading_median0.8526023258587172
deviation-heading_min0.3980684508158633
driven_any_max2.553475024767484
driven_any_mean1.0539409492023473
driven_any_median1.0073534337261636
driven_any_min0.3609526397352217
driven_lanedir_consec_max2.3199839675073
driven_lanedir_consec_mean0.7711579392535753
driven_lanedir_consec_min0.3359040954329422
driven_lanedir_max2.320009778733091
driven_lanedir_mean0.7846552677152641
driven_lanedir_median0.6115512044452298
driven_lanedir_min0.3359040954329422
in-drivable-lane_max6.899999999999996
in-drivable-lane_mean1.2566666666666677
in-drivable-lane_min0
per-episodes
details{"udem1-0-0": {"driven_any": 1.391700043905229, "sim_physics": 0.07472640635019326, "survival_time": 8.299999999999983, "driven_lanedir": 1.3851647080326188, "sim_render-ego": 0.0320512013263013, "in-drivable-lane": 0, "agent_compute-ego": 0.11344962953084922, "deviation-heading": 0.7900143744696351, "set_robot_commands": 0.05085933208465576, "deviation-center-line": 0.531419937088551, "driven_lanedir_consec": 1.384801018742178, "sim_compute_sim_state": 0.02019425903458193, "sim_compute_performance-ego": 0.03522699281393764, "sim_compute_robot_state-ego": 0.03668254111186568, "sim_compute_robot_state-npc0": 0.037042319056499434, "sim_compute_robot_state-npc1": 0.036260652254862955, "sim_compute_robot_state-npc2": 0.035822931542453994, "sim_compute_robot_state-npc3": 0.03556485492062856}, "udem1-1-0": {"driven_any": 0.4483038438306956, "sim_physics": 0.07672588167519405, "survival_time": 2.8999999999999977, "driven_lanedir": 0.4226836157735168, "sim_render-ego": 0.032529978916562834, "in-drivable-lane": 0, "agent_compute-ego": 0.1089727467504041, "deviation-heading": 0.9834096544692096, "set_robot_commands": 0.051750832590563546, "deviation-center-line": 0.2083887780063803, "driven_lanedir_consec": 0.4226836157735168, "sim_compute_sim_state": 0.01996939757774616, "sim_compute_performance-ego": 0.0342238319331202, "sim_compute_robot_state-ego": 0.036735357909367, "sim_compute_robot_state-npc0": 0.036544059884959255, "sim_compute_robot_state-npc1": 0.036746131962743296, "sim_compute_robot_state-npc2": 0.035865578158148406, "sim_compute_robot_state-npc3": 0.035821347401059904}, "udem1-2-0": {"driven_any": 2.1691284131233446, "sim_physics": 0.07492982079001034, "survival_time": 12.750000000000046, "driven_lanedir": 1.7728103279359904, "sim_render-ego": 0.032491230497173236, "in-drivable-lane": 2.2000000000000313, "agent_compute-ego": 0.11285812714520622, "deviation-heading": 0.8643309130348766, "set_robot_commands": 0.050339698791503906, "deviation-center-line": 0.6335547372660157, "driven_lanedir_consec": 1.7727850975910913, "sim_compute_sim_state": 0.02034455467672909, "sim_compute_performance-ego": 0.03525847360199573, "sim_compute_robot_state-ego": 0.03692989162370271, "sim_compute_robot_state-npc0": 0.035881539887072995, "sim_compute_robot_state-npc1": 0.03620606684217266, "sim_compute_robot_state-npc2": 0.03559434928146063, "sim_compute_robot_state-npc3": 0.03572536543303845}, "udem1-3-0": {"driven_any": 2.553475024767484, "sim_physics": 0.07631842692693075, "survival_time": 14.950000000000076, "driven_lanedir": 2.320009778733091, "sim_render-ego": 0.032252218723297116, "in-drivable-lane": 1.3000000000000185, "agent_compute-ego": 0.1128422753016154, "deviation-heading": 0.8526023258587172, "set_robot_commands": 0.049665621916453045, "deviation-center-line": 1.0678270161928525, "driven_lanedir_consec": 2.3199839675073, "sim_compute_sim_state": 0.02014336347579956, "sim_compute_performance-ego": 0.03529156843821208, "sim_compute_robot_state-ego": 0.037095634937286376, "sim_compute_robot_state-npc0": 0.03570375521977742, "sim_compute_robot_state-npc1": 0.03610529104868571, "sim_compute_robot_state-npc2": 0.035761168797810876, "sim_compute_robot_state-npc3": 0.03622866948445638}, "udem1-4-0": {"driven_any": 0.36968771707772213, "sim_physics": 0.07703253201075963, "survival_time": 2.4499999999999993, "driven_lanedir": 0.35661432194998266, "sim_render-ego": 0.0313941021354831, "in-drivable-lane": 0, "agent_compute-ego": 0.1122927714367302, "deviation-heading": 0.6524331044587797, "set_robot_commands": 0.04979231406231316, "deviation-center-line": 0.17497068978210834, "driven_lanedir_consec": 0.35661432194998266, "sim_compute_sim_state": 0.02075913974217006, "sim_compute_performance-ego": 0.0344166950303681, "sim_compute_robot_state-ego": 0.0367360309678681, "sim_compute_robot_state-npc0": 0.036418204404869856, "sim_compute_robot_state-npc1": 0.03638648500247878, "sim_compute_robot_state-npc2": 0.03645906642991669, "sim_compute_robot_state-npc3": 0.03611574854169573}, "udem1-5-0": {"driven_any": 1.042294034462269, "sim_physics": 0.07612440131959461, "survival_time": 6.299999999999986, "driven_lanedir": 0.3977853477095068, "sim_render-ego": 0.032426256982107014, "in-drivable-lane": 2.94999999999999, "agent_compute-ego": 0.11339807321154882, "deviation-heading": 2.060706771196367, "set_robot_commands": 0.0514897864962381, "deviation-center-line": 0.33341663981606123, "driven_lanedir_consec": 0.3977853477095068, "sim_compute_sim_state": 0.0203024565227448, "sim_compute_performance-ego": 0.03575714239998469, "sim_compute_robot_state-ego": 0.03699760777609689, "sim_compute_robot_state-npc0": 0.03603220553625198, "sim_compute_robot_state-npc1": 0.03633174063667419, "sim_compute_robot_state-npc2": 0.0357469225686694, "sim_compute_robot_state-npc3": 0.03591345983838278}, "udem1-6-0": {"driven_any": 0.8064449781297804, "sim_physics": 0.08537807368268871, "survival_time": 4.94999999999999, "driven_lanedir": 0.8035057918243635, "sim_render-ego": 0.032685725375859426, "in-drivable-lane": 0, "agent_compute-ego": 0.11433415220241354, "deviation-heading": 0.4192648649548546, "set_robot_commands": 0.04979162023525045, "deviation-center-line": 0.4521049008644824, "driven_lanedir_consec": 0.8035057918243635, "sim_compute_sim_state": 0.019968651761912336, "sim_compute_performance-ego": 0.0350173001337533, "sim_compute_robot_state-ego": 0.03725468751155969, "sim_compute_robot_state-npc0": 0.03576721567096132, "sim_compute_robot_state-npc1": 0.03590624982660467, "sim_compute_robot_state-npc2": 0.03526998770357383, "sim_compute_robot_state-npc3": 0.03538241049255988}, "udem1-7-0": {"driven_any": 1.0073534337261636, "sim_physics": 0.08103795716019928, "survival_time": 6.099999999999986, "driven_lanedir": 0.8835123128772164, "sim_render-ego": 0.032775183193019176, "in-drivable-lane": 0, "agent_compute-ego": 0.11410353809106544, "deviation-heading": 2.2471929051607855, "set_robot_commands": 0.050079926115567566, "deviation-center-line": 0.3798478400382373, "driven_lanedir_consec": 0.8699557215089209, "sim_compute_sim_state": 0.02000480010861256, "sim_compute_performance-ego": 0.03539958938223417, "sim_compute_robot_state-ego": 0.03694274386421579, "sim_compute_robot_state-npc0": 0.035558735738035106, "sim_compute_robot_state-npc1": 0.03653781922137151, "sim_compute_robot_state-npc2": 0.035670589228145415, "sim_compute_robot_state-npc3": 0.035717876231084106}, "udem1-8-0": {"driven_any": 1.0248237349996017, "sim_physics": 0.07631153445090016, "survival_time": 6.199999999999986, "driven_lanedir": 0.6115512044452298, "sim_render-ego": 0.03288006974804786, "in-drivable-lane": 1.7499999999999938, "agent_compute-ego": 0.11227964778100293, "deviation-heading": 2.0843969428116633, "set_robot_commands": 0.05090678122735793, "deviation-center-line": 0.4228099488267053, "driven_lanedir_consec": 0.6115512044452298, "sim_compute_sim_state": 0.020218643449967908, "sim_compute_performance-ego": 0.0359232694871964, "sim_compute_robot_state-ego": 0.03716015046642673, "sim_compute_robot_state-npc0": 0.03560947410521969, "sim_compute_robot_state-npc1": 0.03608220046566379, "sim_compute_robot_state-npc2": 0.03563206618832004, "sim_compute_robot_state-npc3": 0.03557609550414547}, "udem1-9-0": {"driven_any": 0.44830384366262993, "sim_physics": 0.08678409559973355, "survival_time": 2.8999999999999977, "driven_lanedir": 0.4319199663087821, "sim_render-ego": 0.032657199892504464, "in-drivable-lane": 0, "agent_compute-ego": 0.11270154344624486, "deviation-heading": 0.7843859210886422, "set_robot_commands": 0.04949124517111943, "deviation-center-line": 0.19267161522996132, "driven_lanedir_consec": 0.4319199663087821, "sim_compute_sim_state": 0.020164699390016753, "sim_compute_performance-ego": 0.03502658317829001, "sim_compute_robot_state-ego": 0.03722565338529389, "sim_compute_robot_state-npc0": 0.03597389007436818, "sim_compute_robot_state-npc1": 0.036136195577424146, "sim_compute_robot_state-npc2": 0.035704781269205024, "sim_compute_robot_state-npc3": 0.03587900359055091}, "udem1-10-0": {"driven_any": 0.3609526397352217, "sim_physics": 0.07937508821487427, "survival_time": 2.3999999999999995, "driven_lanedir": 0.3414443288920854, "sim_render-ego": 0.03244545559088389, "in-drivable-lane": 0, "agent_compute-ego": 0.1145672897497813, "deviation-heading": 0.7915362159748539, "set_robot_commands": 0.04971388975779215, "deviation-center-line": 0.16662188238079648, "driven_lanedir_consec": 0.3414443288920854, "sim_compute_sim_state": 0.019728243350982663, "sim_compute_performance-ego": 0.035220980644226074, "sim_compute_robot_state-ego": 0.03813609480857849, "sim_compute_robot_state-npc0": 0.03553371628125509, "sim_compute_robot_state-npc1": 0.03562937180201212, "sim_compute_robot_state-npc2": 0.03530546526114146, "sim_compute_robot_state-npc3": 0.0352599173784256}, "udem1-11-0": {"driven_any": 0.4046281574961264, "sim_physics": 0.07699596656943267, "survival_time": 2.6499999999999986, "driven_lanedir": 0.40003643479537576, "sim_render-ego": 0.03253709145312039, "in-drivable-lane": 0, "agent_compute-ego": 0.11096872473662754, "deviation-heading": 0.3980684508158633, "set_robot_commands": 0.05006668702611383, "deviation-center-line": 0.2708859521467339, "driven_lanedir_consec": 0.40003643479537576, "sim_compute_sim_state": 0.020104664676594285, "sim_compute_performance-ego": 0.03569423027758328, "sim_compute_robot_state-ego": 0.03685567963798091, "sim_compute_robot_state-npc0": 0.036649546533260705, "sim_compute_robot_state-npc1": 0.035206466350915296, "sim_compute_robot_state-npc2": 0.03696849211206976, "sim_compute_robot_state-npc3": 0.035574859043337265}, "udem1-12-0": {"driven_any": 1.5489327489516758, "sim_physics": 0.07977798451548038, "survival_time": 9.199999999999996, "driven_lanedir": 0.3359040954329422, "sim_render-ego": 0.03216033396513566, "in-drivable-lane": 6.899999999999996, "agent_compute-ego": 0.11212264454883078, "deviation-heading": 0.4854799489598317, "set_robot_commands": 0.05000009096187094, "deviation-center-line": 0.2750717398716856, "driven_lanedir_consec": 0.3359040954329422, "sim_compute_sim_state": 0.020557235116543976, "sim_compute_performance-ego": 0.03532131340192712, "sim_compute_robot_state-ego": 0.03672677667244621, "sim_compute_robot_state-npc0": 0.03568084214044654, "sim_compute_robot_state-npc1": 0.03599237099937771, "sim_compute_robot_state-npc2": 0.03542442166286966, "sim_compute_robot_state-npc3": 0.03605055809020996}, "udem1-13-0": {"driven_any": 0.9898831334126, "sim_physics": 0.0766534686088562, "survival_time": 5.999999999999987, "driven_lanedir": 0.6517705220330678, "sim_render-ego": 0.03249402046203613, "in-drivable-lane": 1.4499999999999948, "agent_compute-ego": 0.11407612760861714, "deviation-heading": 1.323248045391386, "set_robot_commands": 0.050547532240549725, "deviation-center-line": 0.3285033071320674, "driven_lanedir_consec": 0.6517705220330678, "sim_compute_sim_state": 0.02088234623273214, "sim_compute_performance-ego": 0.03519213596979777, "sim_compute_robot_state-ego": 0.03679651816685994, "sim_compute_robot_state-npc0": 0.036435035864512126, "sim_compute_robot_state-npc1": 0.03674241105715434, "sim_compute_robot_state-npc2": 0.03615872859954834, "sim_compute_robot_state-npc3": 0.03597831726074219}, "udem1-14-0": {"driven_any": 1.2432024907546595, "sim_physics": 0.08267429851045545, "survival_time": 7.449999999999981, "driven_lanedir": 0.6551162589851884, "sim_render-ego": 0.032208700308063686, "in-drivable-lane": 2.299999999999992, "agent_compute-ego": 0.11307674286349507, "deviation-heading": 3.00376928154286, "set_robot_commands": 0.04945273047325595, "deviation-center-line": 0.4491548536824948, "driven_lanedir_consec": 0.466627654289285, "sim_compute_sim_state": 0.020314541439082, "sim_compute_performance-ego": 0.03505162904726579, "sim_compute_robot_state-ego": 0.036812140637596184, "sim_compute_robot_state-npc0": 0.035848046309195906, "sim_compute_robot_state-npc1": 0.03563666983738842, "sim_compute_robot_state-npc2": 0.03561194791089768, "sim_compute_robot_state-npc3": 0.03557354011791664}}
set_robot_commands_max0.051750832590563546
set_robot_commands_mean0.050263205943373704
set_robot_commands_median0.05006668702611383
set_robot_commands_min0.04945273047325595
sim_compute_performance-ego_max0.0359232694871964
sim_compute_performance-ego_mean0.035201449049326164
sim_compute_performance-ego_median0.03522699281393764
sim_compute_performance-ego_min0.0342238319331202
sim_compute_robot_state-ego_max0.03813609480857849
sim_compute_robot_state-ego_mean0.037005833965142974
sim_compute_robot_state-ego_median0.03692989162370271
sim_compute_robot_state-ego_min0.03668254111186568
sim_compute_robot_state-npc0_max0.037042319056499434
sim_compute_robot_state-npc0_mean0.036045239113779035
sim_compute_robot_state-npc0_median0.035881539887072995
sim_compute_robot_state-npc0_min0.03553371628125509
sim_compute_robot_state-npc1_max0.036746131962743296
sim_compute_robot_state-npc1_mean0.03612707485903531
sim_compute_robot_state-npc1_median0.036136195577424146
sim_compute_robot_state-npc1_min0.035206466350915296
sim_compute_robot_state-npc2_max0.03696849211206976
sim_compute_robot_state-npc2_mean0.03579976644761542
sim_compute_robot_state-npc2_median0.035704781269205024
sim_compute_robot_state-npc2_min0.03526998770357383
sim_compute_robot_state-npc3_max0.03622866948445638
sim_compute_robot_state-npc3_mean0.03575746822188226
sim_compute_robot_state-npc3_median0.03572536543303845
sim_compute_robot_state-npc3_min0.0352599173784256
sim_compute_sim_state_max0.02088234623273214
sim_compute_sim_state_mean0.020243799770414416
sim_compute_sim_state_median0.02019425903458193
sim_compute_sim_state_min0.019728243350982663
sim_physics_max0.08678409559973355
sim_physics_mean0.07872306242568688
sim_physics_median0.07699596656943267
sim_physics_min0.07472640635019326
sim_render-ego_max0.03288006974804786
sim_render-ego_mean0.03239925123797302
sim_render-ego_median0.032491230497173236
sim_render-ego_min0.0313941021354831
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean6.366666666666666
survival_time_min2.3999999999999995
No reset possible
22723step1-simulationhost-erroryes0:57:46
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 544, in get_cr
    uploaded = upload_files(wd, aws_config, copy_to_machine_cache=copy_to_machine_cache)
  File "/project/src/duckietown_challenges_runner/runner.py", line 1123, in upload_files
    uploaded = upload(aws_config, toupload, copy_to_machine_cache=copy_to_machine_cache)
  File "/project/src/duckietown_challenges_runner/runner.py", line 1360, in upload
    copy_to_cache(realfile, sha256hex)
  File "/project/src/duckietown_challenges_runner/runner_cache.py", line 39, in copy_to_cache
    msg = 'Copying %s to cache %s' % (friendly_size(os.stat(fn).st_size), have)
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido2-LFVI-sim-testing/submission2907/step1-simulation-ip-172-31-40-253-12257-job22723/tmp/tmpx34sisis/episodes/udem1-5-0/drawing.html'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
22716step1-simulationsuccessyes0:48:27
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible