Duckietown Challenges Home Challenges Submissions

Submission 10762

Submission10762
Competingyes
Challengeaido5-LF-sim-validation
UserMelisande Teng
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 57913
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

57913

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
57913LFv-simsuccessyes0:08:32
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.1700945377832093
survival_time_median13.450000000000056
deviation-center-line_median0.3043862039984869
in-drivable-lane_median6.725000000000055


other stats
agent_compute-ego0_max0.018971742099865685
agent_compute-ego0_mean0.014040955180103096
agent_compute-ego0_median0.012468870622936596
agent_compute-ego0_min0.012254337374673506
complete-iteration_max0.19821685667960875
complete-iteration_mean0.17169677390215612
complete-iteration_median0.17304427740732087
complete-iteration_min0.14248168411437404
deviation-center-line_max0.7319574158205373
deviation-center-line_mean0.3717492132620064
deviation-center-line_min0.14626702923051443
deviation-heading_max3.0694520387112223
deviation-heading_mean1.6490722715620365
deviation-heading_median1.4247308335486553
deviation-heading_min0.6773753804396139
driven_any_max4.026916219448446
driven_any_mean2.636207038060773
driven_any_median2.5805737191120963
driven_any_min1.3567644945704516
driven_lanedir_consec_max1.9515660775323376
driven_lanedir_consec_mean1.2159243099307204
driven_lanedir_consec_min0.571942086624126
driven_lanedir_max1.9515660775323376
driven_lanedir_mean1.2159243099307204
driven_lanedir_median1.1700945377832093
driven_lanedir_min0.571942086624126
get_duckie_state_max1.2047319527131966e-06
get_duckie_state_mean1.1146266695183028e-06
get_duckie_state_median1.0999895058437748e-06
get_duckie_state_min1.0537957136724652e-06
get_robot_state_max0.003534282442306104
get_robot_state_mean0.0034645026372065652
get_robot_state_median0.0034547185776879234
get_robot_state_min0.003414290951144311
get_state_dump_max0.004276228863656806
get_state_dump_mean0.004227166702916231
get_state_dump_median0.00423296818989919
get_state_dump_min0.00416650156820974
get_ui_image_max0.03502694406817036
get_ui_image_mean0.030025133525943687
get_ui_image_median0.02992349052370187
get_ui_image_min0.025226608988200647
in-drivable-lane_max11.300000000000122
in-drivable-lane_mean7.275000000000054
in-drivable-lane_min4.3499999999999845
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 3.223718654772732, "get_ui_image": 0.028027682866214623, "step_physics": 0.08937045094469882, "survival_time": 16.5000000000001, "driven_lanedir": 1.9515660775323376, "get_state_dump": 0.004204366862593821, "get_robot_state": 0.003534282442306104, "sim_render-ego0": 0.003602477358907371, "get_duckie_state": 1.0537957136724652e-06, "in-drivable-lane": 6.550000000000093, "deviation-heading": 1.6840667698474538, "agent_compute-ego0": 0.018971742099865685, "complete-iteration": 0.1617178578391176, "set_robot_commands": 0.0020718358434579165, "deviation-center-line": 0.44772309586628367, "driven_lanedir_consec": 1.9515660775323376, "sim_compute_sim_state": 0.009984777052957123, "sim_compute_performance-ego0": 0.0018782630067577537}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.3567644945704516, "get_ui_image": 0.03502694406817036, "step_physics": 0.12582375926356162, "survival_time": 7.699999999999981, "driven_lanedir": 0.571942086624126, "get_state_dump": 0.00416650156820974, "get_robot_state": 0.003414290951144311, "sim_render-ego0": 0.00371337398405998, "get_duckie_state": 1.075190882528982e-06, "in-drivable-lane": 4.3499999999999845, "deviation-heading": 1.165394897249857, "agent_compute-ego0": 0.01239626330714072, "complete-iteration": 0.19821685667960875, "set_robot_commands": 0.0021698536411408455, "deviation-center-line": 0.16104931213069013, "driven_lanedir_consec": 0.571942086624126, "sim_compute_sim_state": 0.00954864409662062, "sim_compute_performance-ego0": 0.001884837304392169}, "LF-norm-techtrack-000-ego0": {"driven_any": 4.026916219448446, "get_ui_image": 0.031819298181189115, "step_physics": 0.11440885727664074, "survival_time": 20.70000000000016, "driven_lanedir": 1.713243076186667, "get_state_dump": 0.00426156951720456, "get_robot_state": 0.003457863934068795, "sim_render-ego0": 0.003628661833613752, "get_duckie_state": 1.2047319527131966e-06, "in-drivable-lane": 11.300000000000122, "deviation-heading": 3.0694520387112223, "agent_compute-ego0": 0.012541477938732467, "complete-iteration": 0.18437069697552416, "set_robot_commands": 0.0020600922136421664, "deviation-center-line": 0.7319574158205373, "driven_lanedir_consec": 1.713243076186667, "sim_compute_sim_state": 0.010259362875697123, "sim_compute_performance-ego0": 0.001854852308709937}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.9374287834514612, "get_ui_image": 0.025226608988200647, "step_physics": 0.08482438411438864, "survival_time": 10.400000000000013, "driven_lanedir": 0.6269459993797515, "get_state_dump": 0.004276228863656806, "get_robot_state": 0.003451573221307052, "sim_render-ego0": 0.0036410988803115183, "get_duckie_state": 1.1247881291585676e-06, "in-drivable-lane": 6.900000000000017, "deviation-heading": 0.6773753804396139, "agent_compute-ego0": 0.012254337374673506, "complete-iteration": 0.14248168411437404, "set_robot_commands": 0.0020430156488737993, "deviation-center-line": 0.14626702923051443, "driven_lanedir_consec": 0.6269459993797515, "sim_compute_sim_state": 0.004863263317272423, "sim_compute_performance-ego0": 0.0018264962155282783}}
set_robot_commands_max0.0021698536411408455
set_robot_commands_mean0.0020861993367786817
set_robot_commands_median0.0020659640285500417
set_robot_commands_min0.0020430156488737993
sim_compute_performance-ego0_max0.001884837304392169
sim_compute_performance-ego0_mean0.0018611122088470343
sim_compute_performance-ego0_median0.0018665576577338453
sim_compute_performance-ego0_min0.0018264962155282783
sim_compute_sim_state_max0.010259362875697123
sim_compute_sim_state_mean0.008664011835636825
sim_compute_sim_state_median0.009766710574788874
sim_compute_sim_state_min0.004863263317272423
sim_render-ego0_max0.00371337398405998
sim_render-ego0_mean0.003646403014223155
sim_render-ego0_median0.0036348803569626353
sim_render-ego0_min0.003602477358907371
simulation-passed1
step_physics_max0.12582375926356162
step_physics_mean0.10360686289982246
step_physics_median0.10188965411066978
step_physics_min0.08482438411438864
survival_time_max20.70000000000016
survival_time_mean13.825000000000063
survival_time_min7.699999999999981
No reset possible
51749LFv-simerrorno0:04:05
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140305918089536
- M:video_aido:cmdline(in:/;out:/) 140305918090592
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51741LFv-simerrorno0:06:33
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140692109457008
- M:video_aido:cmdline(in:/;out:/) 140692109459360
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40990LFv-simsuccessno0:08:10
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40989LFv-simsuccessno0:08:13
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
37129LFv-simsuccessno0:09:19
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible