Duckietown Challenges Home Challenges Submissions

Submission 10715

Submission10715
Competingyes
Challengeaido5-LF-sim-validation
UserPhilippe Reddy 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 57978
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

57978

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
57978LFv-simsuccessyes0:23:17
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median2.5133674159640327
survival_time_median52.34999999999916
deviation-center-line_median1.4545687310776263
in-drivable-lane_median7.999999999999924


other stats
agent_compute-ego0_max0.013234244693409314
agent_compute-ego0_mean0.012286720374397745
agent_compute-ego0_median0.012704032441578136
agent_compute-ego0_min0.010504571921025389
complete-iteration_max0.21823893677104603
complete-iteration_mean0.17856262485898794
complete-iteration_median0.18197814013532457
complete-iteration_min0.1320552823942567
deviation-center-line_max1.7486169978057862
deviation-center-line_mean1.2196259601158448
deviation-center-line_min0.22074938050233991
deviation-heading_max9.971057849879031
deviation-heading_mean5.7173373066736435
deviation-heading_median5.429955604359289
deviation-heading_min2.0383801680969618
driven_any_max4.169091248938013
driven_any_mean2.9234677804703493
driven_any_median3.608302097634276
driven_any_min0.30817567767483334
driven_lanedir_consec_max4.069379688125826
driven_lanedir_consec_mean2.310216579763302
driven_lanedir_consec_min0.14475179899931634
driven_lanedir_max4.069379688125826
driven_lanedir_mean2.310216579763302
driven_lanedir_median2.5133674159640327
driven_lanedir_min0.14475179899931634
get_duckie_state_max1.5336042009918383e-06
get_duckie_state_mean1.3504301277271908e-06
get_duckie_state_median1.395283780535781e-06
get_duckie_state_min1.077548748845363e-06
get_robot_state_max0.003962755401763789
get_robot_state_mean0.003663210567155571
get_robot_state_median0.003698985681974567
get_robot_state_min0.0032921155029093595
get_state_dump_max0.005075519444556209
get_state_dump_mean0.004745317404158853
get_state_dump_median0.0048454992208624495
get_state_dump_min0.004214751730354302
get_ui_image_max0.034784186970103874
get_ui_image_mean0.03030379795578183
get_ui_image_median0.03151814071703769
get_ui_image_min0.023394723418948056
in-drivable-lane_max16.24999999999933
in-drivable-lane_mean8.062499999999794
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 3.051004610535862, "get_ui_image": 0.029298331351253572, "step_physics": 0.1107909748674105, "survival_time": 44.6999999999996, "driven_lanedir": 1.8925285398314495, "get_state_dump": 0.005075519444556209, "get_robot_state": 0.003831641234499116, "sim_render-ego0": 0.00403602083302077, "get_duckie_state": 1.5336042009918383e-06, "in-drivable-lane": 16.24999999999933, "deviation-heading": 4.534330322147225, "agent_compute-ego0": 0.012519555118496858, "complete-iteration": 0.1807579853015239, "set_robot_commands": 0.002301662061467517, "deviation-center-line": 1.561678131612248, "driven_lanedir_consec": 1.8925285398314495, "sim_compute_sim_state": 0.010695499015254016, "sim_compute_performance-ego0": 0.0021088802614691537}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.30817567767483334, "get_ui_image": 0.034784186970103874, "step_physics": 0.14330804131247782, "survival_time": 5.449999999999989, "driven_lanedir": 0.14475179899931634, "get_state_dump": 0.004691329869357022, "get_robot_state": 0.003566330129450018, "sim_render-ego0": 0.004010761867869984, "get_duckie_state": 1.322139393199574e-06, "in-drivable-lane": 2.3999999999999915, "deviation-heading": 2.0383801680969618, "agent_compute-ego0": 0.013234244693409314, "complete-iteration": 0.21823893677104603, "set_robot_commands": 0.002220938422463157, "deviation-center-line": 0.22074938050233991, "driven_lanedir_consec": 0.14475179899931634, "sim_compute_sim_state": 0.0103223670612682, "sim_compute_performance-ego0": 0.002002198045903986}, "LF-norm-techtrack-000-ego0": {"driven_any": 4.165599584732689, "get_ui_image": 0.03373795008282181, "step_physics": 0.1079442254907384, "survival_time": 59.99999999999873, "driven_lanedir": 3.134206292096616, "get_state_dump": 0.004999668572367876, "get_robot_state": 0.003962755401763789, "sim_render-ego0": 0.004225850006027285, "get_duckie_state": 1.4684281678719882e-06, "in-drivable-lane": 13.599999999999856, "deviation-heading": 6.3255808865713545, "agent_compute-ego0": 0.012888509764659417, "complete-iteration": 0.18319829496912515, "set_robot_commands": 0.00243946019060705, "deviation-center-line": 1.3474593305430052, "driven_lanedir_consec": 3.134206292096616, "sim_compute_sim_state": 0.01067397814805462, "sim_compute_performance-ego0": 0.0022206169480983662}, "LF-norm-small_loop-000-ego0": {"driven_any": 4.169091248938013, "get_ui_image": 0.023394723418948056, "step_physics": 0.07782037589671113, "survival_time": 59.99999999999873, "driven_lanedir": 4.069379688125826, "get_state_dump": 0.004214751730354302, "get_robot_state": 0.0032921155029093595, "sim_render-ego0": 0.0034806005762181213, "get_duckie_state": 1.077548748845363e-06, "in-drivable-lane": 0.0, "deviation-heading": 9.971057849879031, "agent_compute-ego0": 0.010504571921025389, "complete-iteration": 0.1320552823942567, "set_robot_commands": 0.00194125886165927, "deviation-center-line": 1.7486169978057862, "driven_lanedir_consec": 4.069379688125826, "sim_compute_sim_state": 0.00559685688828747, "sim_compute_performance-ego0": 0.001727035500227065}}
set_robot_commands_max0.00243946019060705
set_robot_commands_mean0.0022258298840492487
set_robot_commands_median0.002261300241965337
set_robot_commands_min0.00194125886165927
sim_compute_performance-ego0_max0.0022206169480983662
sim_compute_performance-ego0_mean0.0020146826889246424
sim_compute_performance-ego0_median0.00205553915368657
sim_compute_performance-ego0_min0.001727035500227065
sim_compute_sim_state_max0.010695499015254016
sim_compute_sim_state_mean0.009322175278216075
sim_compute_sim_state_median0.01049817260466141
sim_compute_sim_state_min0.00559685688828747
sim_render-ego0_max0.004225850006027285
sim_render-ego0_mean0.00393830832078404
sim_render-ego0_median0.004023391350445377
sim_render-ego0_min0.0034806005762181213
simulation-passed1
step_physics_max0.14330804131247782
step_physics_mean0.10996590439183448
step_physics_median0.10936760017907446
step_physics_min0.07782037589671113
survival_time_max59.99999999999873
survival_time_mean42.53749999999926
survival_time_min5.449999999999989
No reset possible
57973LFv-simsuccessyes0:20:20
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
57971LFv-simsuccessyes0:20:54
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
57968LFv-simsuccessyes0:19:05
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51871LFv-simerrorno0:08:40
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139616882471936
- M:video_aido:cmdline(in:/;out:/) 139616882432128
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51868LFv-simerrorno0:07:55
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140372430860240
- M:video_aido:cmdline(in:/;out:/) 140372430872288
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51858LFv-simerrorno0:04:53
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140342296949136
- M:video_aido:cmdline(in:/;out:/) 140342254558128
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51853LFv-simerrorno0:03:26
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140481184039552
- M:video_aido:cmdline(in:/;out:/) 140481182402688
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
41054LFv-simsuccessno0:08:28
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
41053LFv-simsuccessno0:08:32
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
36640LFv-simsuccessno0:12:27
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
36639LFv-simsuccessno0:12:37
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible