Duckietown Challenges Home Challenges Submissions

Submission 11476

Submission11476
Competingyes
Challengeaido5-LF-sim-validation
UserRaphael Jean
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 54515
Next
User labelsim-exercise-2
Admin priority50
Blessingn/a
User priority50

54515

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
54515LFv-simsuccessyes0:33:30
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median10.74920637353492
survival_time_median59.99999999999873
deviation-center-line_median2.576890388448972
in-drivable-lane_median5.649999999999878


other stats
agent_compute-ego0_max0.046890656153361
agent_compute-ego0_mean0.02133298603455424
agent_compute-ego0_median0.012961453343311218
agent_compute-ego0_min0.012518381298233528
complete-iteration_max0.23337732802620537
complete-iteration_mean0.21060283071950517
complete-iteration_median0.2119976248979649
complete-iteration_min0.18503874505588555
deviation-center-line_max2.9587611204624538
deviation-center-line_mean2.148695502039604
deviation-center-line_min0.4822401107980181
deviation-heading_max12.268870868615846
deviation-heading_mean9.173384297375314
deviation-heading_median10.617144284900297
deviation-heading_min3.190377751084814
driven_any_max14.716043368673873
driven_any_mean11.142327390792294
driven_any_median12.082819933954507
driven_any_min5.687626326586289
driven_lanedir_consec_max14.403559559030002
driven_lanedir_consec_mean9.325440558454982
driven_lanedir_consec_min1.3997899277200756
driven_lanedir_max14.403559559030002
driven_lanedir_mean9.332543378836991
driven_lanedir_median10.74920637353492
driven_lanedir_min1.4282012092481169
get_duckie_state_max1.5559740407977075e-06
get_duckie_state_mean1.4319310432852124e-06
get_duckie_state_median1.4241589396124974e-06
get_duckie_state_min1.323432253118147e-06
get_robot_state_max0.004106579573327159
get_robot_state_mean0.003965551660026647
get_robot_state_median0.0039777835143992146
get_robot_state_min0.003800060037981
get_state_dump_max0.005797159462644343
get_state_dump_mean0.005281471342393859
get_state_dump_median0.005118999949700628
get_state_dump_min0.005090726007529837
get_ui_image_max0.036358579410105125
get_ui_image_mean0.03146672908612388
get_ui_image_median0.03163115269536281
get_ui_image_min0.02624603154366477
in-drivable-lane_max20.750000000000274
in-drivable-lane_mean8.012500000000006
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 14.716043368673873, "get_ui_image": 0.029540187214733064, "step_physics": 0.11481213926971208, "survival_time": 59.99999999999873, "driven_lanedir": 14.403559559030002, "get_state_dump": 0.005140179896930374, "get_robot_state": 0.003876503460810246, "sim_render-ego0": 0.003971305516836149, "get_duckie_state": 1.5559740407977075e-06, "in-drivable-lane": 0.0, "deviation-heading": 10.083029118296098, "agent_compute-ego0": 0.012934235907117096, "complete-iteration": 0.18503874505588555, "set_robot_commands": 0.002348947088287633, "deviation-center-line": 2.3401283612050228, "driven_lanedir_consec": 14.403559559030002, "sim_compute_sim_state": 0.010175163005412768, "sim_compute_performance-ego0": 0.0021471151404337126}, "LF-norm-zigzag-000-ego0": {"driven_any": 11.714295880246292, "get_ui_image": 0.036358579410105125, "step_physics": 0.151828386702208, "survival_time": 59.99999999999873, "driven_lanedir": 11.35288186222704, "get_state_dump": 0.005090726007529837, "get_robot_state": 0.004106579573327159, "sim_render-ego0": 0.004161176832391261, "get_duckie_state": 1.379095644478397e-06, "in-drivable-lane": 0.0, "deviation-heading": 12.268870868615846, "agent_compute-ego0": 0.012518381298233528, "complete-iteration": 0.23337732802620537, "set_robot_commands": 0.002433010977968189, "deviation-center-line": 2.9587611204624538, "driven_lanedir_consec": 11.35288186222704, "sim_compute_sim_state": 0.014448659604633976, "sim_compute_performance-ego0": 0.00233668808536069}, "LF-norm-techtrack-000-ego0": {"driven_any": 12.451343987662725, "get_ui_image": 0.03372211817599256, "step_physics": 0.1382628146258123, "survival_time": 59.99999999999873, "driven_lanedir": 10.145530884842804, "get_state_dump": 0.005097820002470882, "get_robot_state": 0.004079063567988183, "sim_render-ego0": 0.004189314790609774, "get_duckie_state": 1.469222234746598e-06, "in-drivable-lane": 11.299999999999756, "deviation-heading": 11.151259451504496, "agent_compute-ego0": 0.01298867077950534, "complete-iteration": 0.216644059212182, "set_robot_commands": 0.0024743778918009814, "deviation-center-line": 2.813652415692921, "driven_lanedir_consec": 10.145530884842804, "sim_compute_sim_state": 0.013409781118515231, "sim_compute_performance-ego0": 0.0023267497428747933}, "LF-norm-small_loop-000-ego0": {"driven_any": 5.687626326586289, "get_ui_image": 0.02624603154366477, "step_physics": 0.10976497792361076, "survival_time": 28.45000000000027, "driven_lanedir": 1.4282012092481169, "get_state_dump": 0.005797159462644343, "get_robot_state": 0.003800060037981, "sim_render-ego0": 0.0038999185227511224, "get_duckie_state": 1.323432253118147e-06, "in-drivable-lane": 20.750000000000274, "deviation-heading": 3.190377751084814, "agent_compute-ego0": 0.046890656153361, "complete-iteration": 0.20735119058374776, "set_robot_commands": 0.002548016581618995, "deviation-center-line": 0.4822401107980181, "driven_lanedir_consec": 1.3997899277200756, "sim_compute_sim_state": 0.006359918912251791, "sim_compute_performance-ego0": 0.001956771131147418}}
set_robot_commands_max0.002548016581618995
set_robot_commands_mean0.0024510881349189494
set_robot_commands_median0.002453694434884585
set_robot_commands_min0.002348947088287633
sim_compute_performance-ego0_max0.00233668808536069
sim_compute_performance-ego0_mean0.0021918310249541533
sim_compute_performance-ego0_median0.0022369324416542527
sim_compute_performance-ego0_min0.001956771131147418
sim_compute_sim_state_max0.014448659604633976
sim_compute_sim_state_mean0.011098380660203442
sim_compute_sim_state_median0.011792472061963998
sim_compute_sim_state_min0.006359918912251791
sim_render-ego0_max0.004189314790609774
sim_render-ego0_mean0.004055428915647077
sim_render-ego0_median0.004066241174613705
sim_render-ego0_min0.0038999185227511224
simulation-passed1
step_physics_max0.151828386702208
step_physics_mean0.12866707963033577
step_physics_median0.12653747694776218
step_physics_min0.10976497792361076
survival_time_max59.99999999999873
survival_time_mean52.11249999999912
survival_time_min28.45000000000027
No reset possible
54513LFv-simsuccessyes0:33:58
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
54510LFv-simsuccessyes0:37:15
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
54509LFv-simsuccessyes0:37:46
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
54508LFv-simsuccessyes0:36:35
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
54503LFv-simsuccessyes0:30:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49686LFv-simerrorno0:08:38
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140089617919232
- M:video_aido:cmdline(in:/;out:/) 140089617835296
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49675LFv-simerrorno0:09:54
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140196083596976
- M:video_aido:cmdline(in:/;out:/) 140196083623968
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49669LFv-simerrorno0:10:17
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140357955867696
- M:video_aido:cmdline(in:/;out:/) 140357955869376
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42517LFv-simsuccessno0:06:44
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42516LFv-simsuccessno0:09:42
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible