Duckietown Challenges Home Challenges Submissions

Submission 2555

Submission2555
Competingyes
Challengeaido2-LFVI-sim-testing
UserAndrea Censi 🇨🇭
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobsstep1-simulation: 22899
Next
User labelchallenge-aido_LF-template-ros - Template solution using ROS
Admin priority50
Blessingn/a
User priority50

The highlights are available only to the owner and the admins.

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
22899step1-simulationsuccessyes0:17:52
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.2875407694053156
survival_time_median2.849999999999998
deviation-center-line_median0.11031194850197353
in-drivable-lane_median0.9499999999999966


other stats
agent_compute-ego_max0.03742229496991193
agent_compute-ego_mean0.03638843934469992
agent_compute-ego_median0.03618979921527937
agent_compute-ego_min0.035584424817284875
deviation-center-line_max0.2701922695523504
deviation-center-line_mean0.13746050030700233
deviation-center-line_min0.07113819587235681
deviation-heading_max1.4854398920276677
deviation-heading_mean0.7236995121312619
deviation-heading_median0.6953408166028192
deviation-heading_min0.3118646573709166
driven_any_max1.934434121703833
driven_any_mean0.806757806634148
driven_any_median0.8072381933001392
driven_any_min0.20912585880423715
driven_lanedir_consec_max0.7222251056746308
driven_lanedir_consec_mean0.3962831867516217
driven_lanedir_consec_min0.1917983424483971
driven_lanedir_max0.7287863177121016
driven_lanedir_mean0.3975640776898326
driven_lanedir_median0.2875407694053156
driven_lanedir_min0.1917983424483971
in-drivable-lane_max4.04999999999999
in-drivable-lane_mean1.2133333333333305
in-drivable-lane_min0
per-episodes
details{"udem1-0-0": {"driven_any": 1.093753441791869, "sim_physics": 0.08035716091293886, "survival_time": 4.149999999999993, "driven_lanedir": 0.4702323960536803, "sim_render-ego": 0.033124561769416534, "in-drivable-lane": 1.6499999999999957, "agent_compute-ego": 0.03629562366439635, "deviation-heading": 1.2111022380750738, "set_robot_commands": 0.04968200534223074, "deviation-center-line": 0.2487133429854821, "driven_lanedir_consec": 0.4702323960536803, "sim_compute_sim_state": 0.02012667311243264, "sim_compute_performance-ego": 0.035464913012033486, "sim_compute_robot_state-ego": 0.03670555999480098, "sim_compute_robot_state-npc0": 0.036115198250276495, "sim_compute_robot_state-npc1": 0.035699591579207456, "sim_compute_robot_state-npc2": 0.035554067198052465, "sim_compute_robot_state-npc3": 0.03559443462325866}, "udem1-1-0": {"driven_any": 0.29795781475632777, "sim_physics": 0.08102898244504575, "survival_time": 1.3500000000000003, "driven_lanedir": 0.2312609677194346, "sim_render-ego": 0.03292521723994502, "in-drivable-lane": 0.05000000000000005, "agent_compute-ego": 0.03742229496991193, "deviation-heading": 0.6953408166028192, "set_robot_commands": 0.0495759646097819, "deviation-center-line": 0.10073232857747254, "driven_lanedir_consec": 0.2312609677194346, "sim_compute_sim_state": 0.0200940591317636, "sim_compute_performance-ego": 0.034829016085024235, "sim_compute_robot_state-ego": 0.03654141779299135, "sim_compute_robot_state-npc0": 0.03691785423843949, "sim_compute_robot_state-npc1": 0.035407949376989295, "sim_compute_robot_state-npc2": 0.03579236842967846, "sim_compute_robot_state-npc3": 0.03599068853590223}, "udem1-2-0": {"driven_any": 1.080947789080493, "sim_physics": 0.08541112679701585, "survival_time": 3.899999999999994, "driven_lanedir": 0.6758329617605024, "sim_render-ego": 0.03318473925957313, "in-drivable-lane": 1.099999999999996, "agent_compute-ego": 0.03701631533793914, "deviation-heading": 1.0268575719271242, "set_robot_commands": 0.04960502416659624, "deviation-center-line": 0.2701922695523504, "driven_lanedir_consec": 0.6735785385729686, "sim_compute_sim_state": 0.02023177880507249, "sim_compute_performance-ego": 0.03534883107894506, "sim_compute_robot_state-ego": 0.037145556547702886, "sim_compute_robot_state-npc0": 0.0360112067980644, "sim_compute_robot_state-npc1": 0.03599198659261068, "sim_compute_robot_state-npc2": 0.03587466631180201, "sim_compute_robot_state-npc3": 0.03572233823629526}, "udem1-3-0": {"driven_any": 0.8072381933001392, "sim_physics": 0.08158576697633978, "survival_time": 2.849999999999998, "driven_lanedir": 0.47292588175405326, "sim_render-ego": 0.032063676599870646, "in-drivable-lane": 0.7999999999999976, "agent_compute-ego": 0.036123271574053845, "deviation-heading": 0.7978002362162198, "set_robot_commands": 0.04996559494420102, "deviation-center-line": 0.13623952041905, "driven_lanedir_consec": 0.47292588175405326, "sim_compute_sim_state": 0.02051862080891927, "sim_compute_performance-ego": 0.03502932766027618, "sim_compute_robot_state-ego": 0.03726878500821298, "sim_compute_robot_state-npc0": 0.03603535786009671, "sim_compute_robot_state-npc1": 0.03562447062709875, "sim_compute_robot_state-npc2": 0.035507331814682276, "sim_compute_robot_state-npc3": 0.03619996706644694}, "udem1-4-0": {"driven_any": 0.31118732113900155, "sim_physics": 0.08198060160097868, "survival_time": 1.1500000000000004, "driven_lanedir": 0.2875407694053156, "sim_render-ego": 0.03197870047196098, "in-drivable-lane": 0, "agent_compute-ego": 0.03628433269003163, "deviation-heading": 0.39989969254594865, "set_robot_commands": 0.049159817073656166, "deviation-center-line": 0.07144385614371519, "driven_lanedir_consec": 0.2875407694053156, "sim_compute_sim_state": 0.01976142758908479, "sim_compute_performance-ego": 0.034766072812287704, "sim_compute_robot_state-ego": 0.036811631658802864, "sim_compute_robot_state-npc0": 0.0351786820784859, "sim_compute_robot_state-npc1": 0.03617349914882494, "sim_compute_robot_state-npc2": 0.035586916882058846, "sim_compute_robot_state-npc3": 0.035502381946729576}, "udem1-5-0": {"driven_any": 1.3021233152606932, "sim_physics": 0.08073924569522634, "survival_time": 5.09999999999999, "driven_lanedir": 0.20473092327823528, "sim_render-ego": 0.03248116783067292, "in-drivable-lane": 4.04999999999999, "agent_compute-ego": 0.03599894981758267, "deviation-heading": 0.394574000309935, "set_robot_commands": 0.05010882078432569, "deviation-center-line": 0.10111471927467218, "driven_lanedir_consec": 0.20473092327823528, "sim_compute_sim_state": 0.020437993255316044, "sim_compute_performance-ego": 0.03527142487320246, "sim_compute_robot_state-ego": 0.03728824970768947, "sim_compute_robot_state-npc0": 0.036380704711465275, "sim_compute_robot_state-npc1": 0.03583891718995338, "sim_compute_robot_state-npc2": 0.03576194538789637, "sim_compute_robot_state-npc3": 0.035927195175021305}, "udem1-6-0": {"driven_any": 1.934434121703833, "sim_physics": 0.07922847781862531, "survival_time": 6.999999999999983, "driven_lanedir": 0.6746993221948967, "sim_render-ego": 0.03256455319268363, "in-drivable-lane": 3.899999999999988, "agent_compute-ego": 0.036003969396863664, "deviation-heading": 1.4854398920276677, "set_robot_commands": 0.05034008877617972, "deviation-center-line": 0.2528639324079792, "driven_lanedir_consec": 0.6740101897237831, "sim_compute_sim_state": 0.01994077478136335, "sim_compute_performance-ego": 0.034859599385942734, "sim_compute_robot_state-ego": 0.036635584490639826, "sim_compute_robot_state-npc0": 0.03551873649869646, "sim_compute_robot_state-npc1": 0.03626098121915545, "sim_compute_robot_state-npc2": 0.03612312759671892, "sim_compute_robot_state-npc3": 0.035678916318075994}, "udem1-7-0": {"driven_any": 1.062597738241834, "sim_physics": 0.08417885924038822, "survival_time": 3.649999999999995, "driven_lanedir": 0.7287863177121016, "sim_render-ego": 0.033338608807080415, "in-drivable-lane": 0.7999999999999972, "agent_compute-ego": 0.036171178295187754, "deviation-heading": 0.7849150213204695, "set_robot_commands": 0.048654706510778974, "deviation-center-line": 0.1387309750370881, "driven_lanedir_consec": 0.7222251056746308, "sim_compute_sim_state": 0.02019320448783979, "sim_compute_performance-ego": 0.03470016179019458, "sim_compute_robot_state-ego": 0.03662954617853034, "sim_compute_robot_state-npc0": 0.03585578970713158, "sim_compute_robot_state-npc1": 0.03622668410000736, "sim_compute_robot_state-npc2": 0.035563341558796084, "sim_compute_robot_state-npc3": 0.03537048052435052}, "udem1-8-0": {"driven_any": 1.1602870725462493, "sim_physics": 0.07614908939184145, "survival_time": 4.299999999999993, "driven_lanedir": 0.35913090517817636, "sim_render-ego": 0.03238085813300554, "in-drivable-lane": 2.399999999999996, "agent_compute-ego": 0.035584424817284875, "deviation-heading": 1.2197696860883536, "set_robot_commands": 0.04925437860710676, "deviation-center-line": 0.15216623639979054, "driven_lanedir_consec": 0.35913090517817636, "sim_compute_sim_state": 0.02023070634797562, "sim_compute_performance-ego": 0.03496678485426792, "sim_compute_robot_state-ego": 0.03726999704227891, "sim_compute_robot_state-npc0": 0.036080948142118235, "sim_compute_robot_state-npc1": 0.035573890042859456, "sim_compute_robot_state-npc2": 0.035724426424780555, "sim_compute_robot_state-npc3": 0.03547574198523233}, "udem1-9-0": {"driven_any": 0.2911508182793437, "sim_physics": 0.0902637059871967, "survival_time": 1.3000000000000005, "driven_lanedir": 0.2409218388122456, "sim_render-ego": 0.034405763332660384, "in-drivable-lane": 0.05000000000000005, "agent_compute-ego": 0.03708419433006874, "deviation-heading": 0.5613432181706876, "set_robot_commands": 0.04944897614992582, "deviation-center-line": 0.0760477654868993, "driven_lanedir_consec": 0.2409218388122456, "sim_compute_sim_state": 0.020515955411470853, "sim_compute_performance-ego": 0.03589354111598088, "sim_compute_robot_state-ego": 0.03766464270078219, "sim_compute_robot_state-npc0": 0.03635778793921837, "sim_compute_robot_state-npc1": 0.03625757877643292, "sim_compute_robot_state-npc2": 0.03604723856999324, "sim_compute_robot_state-npc3": 0.03682374954223633}, "udem1-10-0": {"driven_any": 1.021491996160475, "sim_physics": 0.07627735455830892, "survival_time": 3.7499999999999942, "driven_lanedir": 0.7026537170655107, "sim_render-ego": 0.0331515375773112, "in-drivable-lane": 0.9499999999999966, "agent_compute-ego": 0.03578185717264811, "deviation-heading": 0.7506445024068277, "set_robot_commands": 0.05011655489603679, "deviation-center-line": 0.11031194850197353, "driven_lanedir_consec": 0.6929451206884647, "sim_compute_sim_state": 0.02048972447713216, "sim_compute_performance-ego": 0.03607481320699056, "sim_compute_robot_state-ego": 0.03696616808573405, "sim_compute_robot_state-npc0": 0.03663914362589518, "sim_compute_robot_state-npc1": 0.035787041982014975, "sim_compute_robot_state-npc2": 0.03595687866210937, "sim_compute_robot_state-npc3": 0.03647539774576823}, "udem1-11-0": {"driven_any": 0.20912585880423715, "sim_physics": 0.08107545262291319, "survival_time": 1.0500000000000005, "driven_lanedir": 0.1917983424483971, "sim_render-ego": 0.03523479189191546, "in-drivable-lane": 0, "agent_compute-ego": 0.0366175061180478, "deviation-heading": 0.3455718910535376, "set_robot_commands": 0.04895338558015369, "deviation-center-line": 0.10333747877015344, "driven_lanedir_consec": 0.1917983424483971, "sim_compute_sim_state": 0.0201894896371024, "sim_compute_performance-ego": 0.03483440762474423, "sim_compute_robot_state-ego": 0.03691003436133975, "sim_compute_robot_state-npc0": 0.036101182301839195, "sim_compute_robot_state-npc1": 0.03586201440720331, "sim_compute_robot_state-npc2": 0.03554148901076544, "sim_compute_robot_state-npc3": 0.035480101903279625}, "udem1-12-0": {"driven_any": 0.5918906368569831, "sim_physics": 0.08145460971566133, "survival_time": 2.1500000000000004, "driven_lanedir": 0.19652140216443992, "sim_render-ego": 0.03230021166246991, "in-drivable-lane": 1.15, "agent_compute-ego": 0.036010187725688134, "deviation-heading": 0.32606788824321575, "set_robot_commands": 0.049340259197146394, "deviation-center-line": 0.10664063848740413, "driven_lanedir_consec": 0.19652140216443992, "sim_compute_sim_state": 0.02007085223530614, "sim_compute_performance-ego": 0.03510226205337879, "sim_compute_robot_state-ego": 0.03774099017298499, "sim_compute_robot_state-npc0": 0.035571270210798395, "sim_compute_robot_state-npc1": 0.03568903789963833, "sim_compute_robot_state-npc2": 0.03596586959306584, "sim_compute_robot_state-npc3": 0.035720841829166856}, "udem1-13-0": {"driven_any": 0.2815655086390766, "sim_physics": 0.07910509620394025, "survival_time": 1.4000000000000006, "driven_lanedir": 0.27004610417980546, "sim_render-ego": 0.033354265349251885, "in-drivable-lane": 0, "agent_compute-ego": 0.03724268504551479, "deviation-heading": 0.3118646573709166, "set_robot_commands": 0.04872101545333862, "deviation-center-line": 0.12223429668864684, "driven_lanedir_consec": 0.27004610417980546, "sim_compute_sim_state": 0.020237437316349575, "sim_compute_performance-ego": 0.03456952742167881, "sim_compute_robot_state-ego": 0.03654638358524868, "sim_compute_robot_state-npc0": 0.03564871209008353, "sim_compute_robot_state-npc1": 0.03666795151574271, "sim_compute_robot_state-npc2": 0.03581427676337106, "sim_compute_robot_state-npc3": 0.03531798294612339}, "udem1-14-0": {"driven_any": 0.655615472951668, "sim_physics": 0.08990004950878668, "survival_time": 2.549999999999999, "driven_lanedir": 0.25637931562069505, "sim_render-ego": 0.03241805936775956, "in-drivable-lane": 1.2999999999999985, "agent_compute-ego": 0.03618979921527937, "deviation-heading": 0.5443013696101298, "set_robot_commands": 0.0500799861608767, "deviation-center-line": 0.07113819587235681, "driven_lanedir_consec": 0.25637931562069505, "sim_compute_sim_state": 0.02028620944303625, "sim_compute_performance-ego": 0.03464384172476974, "sim_compute_robot_state-ego": 0.03777953690173579, "sim_compute_robot_state-npc0": 0.03697453760633282, "sim_compute_robot_state-npc1": 0.035994221182430494, "sim_compute_robot_state-npc2": 0.03542656991996017, "sim_compute_robot_state-npc3": 0.03537311273462632}}
set_robot_commands_max0.05034008877617972
set_robot_commands_mean0.049533771883489015
set_robot_commands_median0.0495759646097819
set_robot_commands_min0.048654706510778974
sim_compute_performance-ego_max0.03607481320699056
sim_compute_performance-ego_mean0.03509030164664782
sim_compute_performance-ego_median0.03496678485426792
sim_compute_performance-ego_min0.03456952742167881
sim_compute_robot_state-ego_max0.03777953690173579
sim_compute_robot_state-ego_mean0.037060272281965
sim_compute_robot_state-ego_median0.03696616808573405
sim_compute_robot_state-ego_min0.03654141779299135
sim_compute_robot_state-npc0_max0.03697453760633282
sim_compute_robot_state-npc0_mean0.036092474137262805
sim_compute_robot_state-npc0_median0.036080948142118235
sim_compute_robot_state-npc0_min0.0351786820784859
sim_compute_robot_state-npc1_max0.03666795151574271
sim_compute_robot_state-npc1_mean0.03593705437601131
sim_compute_robot_state-npc1_median0.03586201440720331
sim_compute_robot_state-npc1_min0.035407949376989295
sim_compute_robot_state-npc2_max0.03612312759671892
sim_compute_robot_state-npc2_mean0.03574936760824874
sim_compute_robot_state-npc2_median0.03576194538789637
sim_compute_robot_state-npc2_min0.03542656991996017
sim_compute_robot_state-npc3_max0.03682374954223633
sim_compute_robot_state-npc3_mean0.03577688874083424
sim_compute_robot_state-npc3_median0.035678916318075994
sim_compute_robot_state-npc3_min0.03531798294612339
sim_compute_sim_state_max0.02051862080891927
sim_compute_sim_state_mean0.020221660456011
sim_compute_sim_state_median0.02023070634797562
sim_compute_sim_state_min0.01976142758908479
sim_physics_max0.0902637059871967
sim_physics_mean0.08191570529834717
sim_physics_median0.08107545262291319
sim_physics_min0.07614908939184145
sim_render-ego_max0.03523479189191546
sim_render-ego_mean0.03299378083237182
sim_render-ego_median0.03292521723994502
sim_render-ego_min0.03197870047196098
simulation-passed1
survival_time_max6.999999999999983
survival_time_mean3.043333333333329
survival_time_min1.0500000000000005
No reset possible
21729step1-simulationsuccessno0:31:32
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
21724step1-simulationsuccessno0:28:03
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
21710step1-simulationhost-errorno0:25:10
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 488, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
    next(self.gen)
  File "/project/src/duckietown_challenges_runner/runner.py", line 347, in setup_logging
    convert(f_stdout)
  File "/project/src/duckietown_challenges_runner/runner.py", line 343, in convert
    with open(fn, 'w') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido2-LFVI-sim-testing/submission2555/step1-simulation-ip-172-31-25-98-2348-job21710/logs/challenges-runner/stdout.html'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
21189step1-simulationsuccessno0:20:04
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20482step1-simulationsuccessno0:14:21
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
19967step1-simulationsuccessno0:07:51
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
19789step1-simulationsuccessno0:08:42
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
19521step1-simulationsuccessno0:06:31
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18797step1-simulationsuccessno0:13:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18782step1-simulationhost-errorno0:09:13
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 990, in run_docker
    return subprocess.check_output(cmd0, cwd=cwd, stderr=sys.stderr)
  File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker-compose', '-p', 'job18782-372170', 'ps', '-q', 'solution']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 478, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/project/src/duckietown_challenges_runner/runner.py", line 573, in run_single
    write_logs(wd, project, services=config['services'])
  File "/project/src/duckietown_challenges_runner/runner.py", line 899, in write_logs
    container_id = get_id_for_service(wd, project, service)
  File "/project/src/duckietown_challenges_runner/runner.py", line 585, in get_id_for_service
    o = run_docker(wd, project, cmd, get_output=True)
  File "/project/src/duckietown_challenges_runner/runner.py", line 998, in run_docker
    raise DockerComposeFail(msg) from e
duckietown_challenges_runner.runner.DockerComposeFail: Could not run ['docker-compose', '-p', 'job18782-372170', 'ps', '-q', 'solution']:

   >  Command '['docker-compose', '-p', 'job18782-372170', 'ps', '-q', 'solution']' returned non-zero exit status 1.

stdout | 

stderr | 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18781step1-simulationsuccessno0:12:27
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18419step1-simulationsuccessno0:10:11
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
17982step1-simulationsuccessno0:06:23
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
17978step1-simulationsuccessno0:07:26
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible