Duckietown Challenges Home Challenges Submissions

Submission 10890

Submission10890
Competingyes
Challengeaido5-LF-sim-validation
UserDishank Bansal 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 57583
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

57583

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
57583LFv-simsuccessyes0:22:23
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.7679691634889168
survival_time_median45.59999999999952
deviation-center-line_median0.2654819360929188
in-drivable-lane_median35.474999999999525


other stats
agent_compute-ego0_max0.013836056900024414
agent_compute-ego0_mean0.013422599627480976
agent_compute-ego0_median0.013420833040534407
agent_compute-ego0_min0.01301267552883067
complete-iteration_max0.23876268305677048
complete-iteration_mean0.19473812590540143
complete-iteration_median0.18648276524682725
complete-iteration_min0.16722429007118092
deviation-center-line_max0.7272459295850703
deviation-center-line_mean0.3455490814274025
deviation-center-line_min0.12398652393870208
deviation-heading_max3.940737046902254
deviation-heading_mean2.272645675090356
deviation-heading_median2.154635113045895
deviation-heading_min0.8405754273673791
driven_any_max10.422624374347471
driven_any_mean6.6840235911838946
driven_any_median7.84140445537491
driven_any_min0.6306610796382838
driven_lanedir_consec_max2.091389035911053
driven_lanedir_consec_mean0.9601430972661912
driven_lanedir_consec_min0.2132450261758776
driven_lanedir_max2.1128170405828812
driven_lanedir_mean0.965500098434148
driven_lanedir_median0.7679691634889168
driven_lanedir_min0.2132450261758776
get_duckie_state_max1.4621734619140624e-06
get_duckie_state_mean1.3528213882466565e-06
get_duckie_state_median1.3620232066742884e-06
get_duckie_state_min1.225065677723986e-06
get_robot_state_max0.003984148357432649
get_robot_state_mean0.003880587906841434
get_robot_state_median0.0038749256286494047
get_robot_state_min0.0037883520126342778
get_state_dump_max0.0049695823669433595
get_state_dump_mean0.0048072176550945376
get_state_dump_median0.004783244801996947
get_state_dump_min0.004692798649440896
get_ui_image_max0.036792334089887906
get_ui_image_mean0.03191178378949189
get_ui_image_median0.031230522065873352
get_ui_image_min0.02839375693633296
in-drivable-lane_max55.29999999999873
in-drivable-lane_mean32.28749999999944
in-drivable-lane_min2.8999999999999906
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 10.42260503733178, "get_ui_image": 0.029223962985506464, "step_physics": 0.1084982000917916, "survival_time": 59.99999999999873, "driven_lanedir": 2.1128170405828812, "get_state_dump": 0.004692798649440896, "get_robot_state": 0.003833333618138652, "sim_render-ego0": 0.004014954975900007, "get_duckie_state": 1.4166153043037053e-06, "in-drivable-lane": 46.19999999999875, "deviation-heading": 3.940737046902254, "agent_compute-ego0": 0.013204256561177656, "complete-iteration": 0.1771079983739035, "set_robot_commands": 0.0022567825253857463, "deviation-center-line": 0.7272459295850703, "driven_lanedir_consec": 2.091389035911053, "sim_compute_sim_state": 0.009206569760566347, "sim_compute_performance-ego0": 0.002088548738096874}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.6306610796382838, "get_ui_image": 0.036792334089887906, "step_physics": 0.16144102938631746, "survival_time": 4.6499999999999915, "driven_lanedir": 0.2132450261758776, "get_state_dump": 0.004715323448181152, "get_robot_state": 0.0037883520126342778, "sim_render-ego0": 0.0040078949421010115, "get_duckie_state": 1.225065677723986e-06, "in-drivable-lane": 2.8999999999999906, "deviation-heading": 0.8966834010812742, "agent_compute-ego0": 0.01301267552883067, "complete-iteration": 0.23876268305677048, "set_robot_commands": 0.0022517417339568443, "deviation-center-line": 0.12398652393870208, "driven_lanedir_consec": 0.2132450261758776, "sim_compute_sim_state": 0.010664752189149248, "sim_compute_performance-ego0": 0.0020057419513134244}, "LF-norm-techtrack-000-ego0": {"driven_any": 5.260203873418041, "get_ui_image": 0.03323708114624024, "step_physics": 0.12106099815368652, "survival_time": 31.20000000000031, "driven_lanedir": 0.7969076668079387, "get_state_dump": 0.0049695823669433595, "get_robot_state": 0.003916517639160157, "sim_render-ego0": 0.00409962387084961, "get_duckie_state": 1.4621734619140624e-06, "in-drivable-lane": 24.750000000000295, "deviation-heading": 3.4125868250105165, "agent_compute-ego0": 0.013836056900024414, "complete-iteration": 0.195857532119751, "set_robot_commands": 0.002416288757324219, "deviation-center-line": 0.3278814866972835, "driven_lanedir_consec": 0.7969076668079387, "sim_compute_sim_state": 0.010090781021118165, "sim_compute_performance-ego0": 0.0021280872344970705}, "LF-norm-small_loop-000-ego0": {"driven_any": 10.422624374347471, "get_ui_image": 0.02839375693633296, "step_physics": 0.101252109184551, "survival_time": 59.99999999999873, "driven_lanedir": 0.7390306601698948, "get_state_dump": 0.0048511661558127426, "get_robot_state": 0.003984148357432649, "sim_render-ego0": 0.00410627445313059, "get_duckie_state": 1.3074311090448714e-06, "in-drivable-lane": 55.29999999999873, "deviation-heading": 0.8405754273673791, "agent_compute-ego0": 0.01363740951989116, "complete-iteration": 0.16722429007118092, "set_robot_commands": 0.002393655435528783, "deviation-center-line": 0.2030823854885541, "driven_lanedir_consec": 0.7390306601698948, "sim_compute_sim_state": 0.00645246732046364, "sim_compute_performance-ego0": 0.002066451048076798}}
set_robot_commands_max0.002416288757324219
set_robot_commands_mean0.0023296171130488983
set_robot_commands_median0.0023252189804572644
set_robot_commands_min0.0022517417339568443
sim_compute_performance-ego0_max0.0021280872344970705
sim_compute_performance-ego0_mean0.0020722072429960416
sim_compute_performance-ego0_median0.0020774998930868363
sim_compute_performance-ego0_min0.0020057419513134244
sim_compute_sim_state_max0.010664752189149248
sim_compute_sim_state_mean0.00910364257282435
sim_compute_sim_state_median0.009648675390842257
sim_compute_sim_state_min0.00645246732046364
sim_render-ego0_max0.00410627445313059
sim_render-ego0_mean0.004057187060495304
sim_render-ego0_median0.004057289423374808
sim_render-ego0_min0.0040078949421010115
simulation-passed1
step_physics_max0.16144102938631746
step_physics_mean0.12306308420408664
step_physics_median0.11477959912273904
step_physics_min0.101252109184551
survival_time_max59.99999999999873
survival_time_mean38.96249999999944
survival_time_min4.6499999999999915
No reset possible
57579LFv-simtimeoutyes----No reset possible
57576LFv-simsuccessyes0:24:21
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
57570LFv-simtimeoutyes----No reset possible
51342LFv-simerrorno0:03:13
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139971354128288
- M:video_aido:cmdline(in:/;out:/) 139971353266112
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51329LFv-simerrorno0:07:20
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140240173293472
- M:video_aido:cmdline(in:/;out:/) 140240173293232
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40651LFv-simsuccessno0:08:05
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40650LFv-simsuccessno0:09:20
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38354LFv-simsuccessno0:08:11
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible