Duckietown Challenges Home Challenges Submissions

Submission 11389

Submission11389
Competingyes
Challengeaido5-LF-sim-validation
UserYishu Malhotra 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 54672
Next
User labelsim-exercise-1
Admin priority50
Blessingn/a
User priority50

54672

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
54672LFv-simsuccessyes0:14:27
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.7061854960342252
survival_time_median19.550000000000143
deviation-center-line_median0.1824837990580079
in-drivable-lane_median16.450000000000145


other stats
agent_compute-ego0_max0.012193637558176549
agent_compute-ego0_mean0.011956945276417129
agent_compute-ego0_median0.011891551045129827
agent_compute-ego0_min0.011851041457232307
complete-iteration_max0.19721441601847745
complete-iteration_mean0.19020977511761836
complete-iteration_median0.19230539719942324
complete-iteration_min0.17901389005314952
deviation-center-line_max1.2209512761056331
deviation-center-line_mean0.43016699184616314
deviation-center-line_min0.13474909316300324
deviation-heading_max4.457952337450719
deviation-heading_mean1.9367712557594312
deviation-heading_median1.3494001792082095
deviation-heading_min0.5903323271705876
driven_any_max10.355541661981336
driven_any_mean4.926442041865494
driven_any_median3.640993726506741
driven_any_min2.0682390524671614
driven_lanedir_consec_max3.851857540061628
driven_lanedir_consec_mean1.3900862778761702
driven_lanedir_consec_min0.2961165793746021
driven_lanedir_max3.851857540061628
driven_lanedir_mean1.3973027695922522
driven_lanedir_median0.7061854960342252
driven_lanedir_min0.3249825462389295
get_duckie_state_max2.000905290434632e-06
get_duckie_state_mean1.8778512970380503e-06
get_duckie_state_median1.9200836908883114e-06
get_duckie_state_min1.670332515940947e-06
get_robot_state_max0.003702766281642994
get_robot_state_mean0.0035655799497529466
get_robot_state_median0.0036267011898681546
get_robot_state_min0.003306151137632482
get_state_dump_max0.004723674507551296
get_state_dump_mean0.004490016996194263
get_state_dump_median0.004524595161190679
get_state_dump_min0.0041872031548443964
get_ui_image_max0.03136414289474487
get_ui_image_mean0.02875785157056783
get_ui_image_median0.02885946955177819
get_ui_image_min0.025948324283970058
in-drivable-lane_max28.899999999999384
in-drivable-lane_mean17.637499999999925
in-drivable-lane_min8.750000000000036
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 10.355541661981336, "get_ui_image": 0.028038523786811417, "step_physics": 0.12452705085918468, "survival_time": 46.4499999999995, "driven_lanedir": 3.851857540061628, "get_state_dump": 0.004723674507551296, "get_robot_state": 0.003682735658461048, "sim_render-ego0": 0.003724400715161395, "get_duckie_state": 1.987590584703671e-06, "in-drivable-lane": 28.899999999999384, "deviation-heading": 4.457952337450719, "agent_compute-ego0": 0.011927948203138123, "complete-iteration": 0.1910143647142636, "set_robot_commands": 0.0021755533833657543, "deviation-center-line": 1.2209512761056331, "driven_lanedir_consec": 3.851857540061628, "sim_compute_sim_state": 0.010162879574683405, "sim_compute_performance-ego0": 0.00196685791015625}, "LF-norm-zigzag-000-ego0": {"driven_any": 3.2653705268281756, "get_ui_image": 0.03136414289474487, "step_physics": 0.12643355622011074, "survival_time": 16.950000000000106, "driven_lanedir": 0.3249825462389295, "get_state_dump": 0.0041872031548443964, "get_robot_state": 0.003306151137632482, "sim_render-ego0": 0.003437634075389189, "get_duckie_state": 1.670332515940947e-06, "in-drivable-lane": 13.900000000000109, "deviation-heading": 1.899059160784163, "agent_compute-ego0": 0.011851041457232307, "complete-iteration": 0.19359642968458288, "set_robot_commands": 0.0020233876564923455, "deviation-center-line": 0.22411691358891017, "driven_lanedir_consec": 0.2961165793746021, "sim_compute_sim_state": 0.009145654650295482, "sim_compute_performance-ego0": 0.001770800001481}, "LF-norm-techtrack-000-ego0": {"driven_any": 4.016616926185306, "get_ui_image": 0.02968041531674497, "step_physics": 0.12981102112177256, "survival_time": 22.15000000000018, "driven_lanedir": 0.7371447647111731, "get_state_dump": 0.00447695373414873, "get_robot_state": 0.003570666721275261, "sim_render-ego0": 0.0036099273879248816, "get_duckie_state": 1.852576797072952e-06, "in-drivable-lane": 19.000000000000185, "deviation-heading": 0.5903323271705876, "agent_compute-ego0": 0.011855153887121527, "complete-iteration": 0.19721441601847745, "set_robot_commands": 0.0020984185708535683, "deviation-center-line": 0.1408506845271057, "driven_lanedir_consec": 0.7371447647111731, "sim_compute_sim_state": 0.010145918206051664, "sim_compute_performance-ego0": 0.001883478852005692}, "LF-norm-small_loop-000-ego0": {"driven_any": 2.0682390524671614, "get_ui_image": 0.025948324283970058, "step_physics": 0.11908256554905372, "survival_time": 11.800000000000033, "driven_lanedir": 0.6752262273572773, "get_state_dump": 0.004572236588232628, "get_robot_state": 0.003702766281642994, "sim_render-ego0": 0.003862170730462054, "get_duckie_state": 2.000905290434632e-06, "in-drivable-lane": 8.750000000000036, "deviation-heading": 0.7997411976322562, "agent_compute-ego0": 0.012193637558176549, "complete-iteration": 0.17901389005314952, "set_robot_commands": 0.002250558716335377, "deviation-center-line": 0.13474909316300324, "driven_lanedir_consec": 0.6752262273572773, "sim_compute_sim_state": 0.0053354496694315336, "sim_compute_performance-ego0": 0.0019826919217652912}}
set_robot_commands_max0.002250558716335377
set_robot_commands_mean0.002136979581761762
set_robot_commands_median0.0021369859771096615
set_robot_commands_min0.0020233876564923455
sim_compute_performance-ego0_max0.0019826919217652912
sim_compute_performance-ego0_mean0.0019009571713520584
sim_compute_performance-ego0_median0.0019251683810809711
sim_compute_performance-ego0_min0.001770800001481
sim_compute_sim_state_max0.010162879574683405
sim_compute_sim_state_mean0.00869747552511552
sim_compute_sim_state_median0.009645786428173571
sim_compute_sim_state_min0.0053354496694315336
sim_render-ego0_max0.003862170730462054
sim_render-ego0_mean0.00365853322723438
sim_render-ego0_median0.0036671640515431383
sim_render-ego0_min0.003437634075389189
simulation-passed1
step_physics_max0.12981102112177256
step_physics_mean0.12496354843753044
step_physics_median0.1254803035396477
step_physics_min0.11908256554905372
survival_time_max46.4499999999995
survival_time_mean24.337499999999956
survival_time_min11.800000000000033
No reset possible
54666LFv-simsuccessyes0:10:34
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
54663LFv-simsuccessyes0:26:44
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49859LFv-simerrorno0:03:40
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140430642910256
- M:video_aido:cmdline(in:/;out:/) 140430643038336
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49832LFv-simhost-errorno0:05:22
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42178LFv-simsuccessno0:09:32
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42177LFv-simsuccessno0:09:34
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42176LFv-simsuccessno0:08:27
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42175LFv-simsuccessno0:08:14
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible