Duckietown Challenges Home Challenges Submissions

Submission 10997

Submission10997
Competingyes
Challengeaido5-LF-sim-validation
UserMo Kleit 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 57185
Next
User labelsim-exercise-1
Admin priority50
Blessingn/a
User priority50

57185

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
57185LFv-simsuccessyes0:27:41
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median3.5873352510485708
survival_time_median43.74999999999949
deviation-center-line_median1.8282341487568563
in-drivable-lane_median8.300000000000118


other stats
agent_compute-ego0_max0.01489896306974269
agent_compute-ego0_mean0.013827970984902072
agent_compute-ego0_median0.013792257112731058
agent_compute-ego0_min0.012828406644403488
complete-iteration_max0.2390783045123147
complete-iteration_mean0.1941807412292337
complete-iteration_median0.19037344273145168
complete-iteration_min0.15689777494171675
deviation-center-line_max5.192180066600432
deviation-center-line_mean2.5141853979180313
deviation-center-line_min1.2080932275579792
deviation-heading_max12.686158690992796
deviation-heading_mean6.866565249434585
deviation-heading_median6.198145466402819
deviation-heading_min2.3838113739399076
driven_any_max7.063567132962424
driven_any_mean4.528043799318217
driven_any_median4.478118587430645
driven_any_min2.092370889449156
driven_lanedir_consec_max6.8538346452510535
driven_lanedir_consec_mean3.817184690407263
driven_lanedir_consec_min1.2402336142808588
driven_lanedir_max6.8538346452510535
driven_lanedir_mean3.817184690407263
driven_lanedir_median3.5873352510485708
driven_lanedir_min1.2402336142808588
get_duckie_state_max1.811857987435905e-06
get_duckie_state_mean1.5912507147857608e-06
get_duckie_state_median1.6075710939948969e-06
get_duckie_state_min1.3380026837173448e-06
get_robot_state_max0.0044250919529325895
get_robot_state_mean0.004099026306135192
get_robot_state_median0.004102514217719774
get_robot_state_min0.0037659848361686304
get_state_dump_max0.005391522587423793
get_state_dump_mean0.005089872325020847
get_state_dump_median0.005064491366901838
get_state_dump_min0.004838983978855918
get_ui_image_max0.03964996121539827
get_ui_image_mean0.03265439535176106
get_ui_image_median0.03225037247032866
get_ui_image_min0.026466875250988654
in-drivable-lane_max18.749999999999044
in-drivable-lane_mean8.837499999999821
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 2.4486973664419307, "get_ui_image": 0.0311199411249284, "step_physics": 0.11504077849770084, "survival_time": 19.30000000000014, "driven_lanedir": 2.1675671005500683, "get_state_dump": 0.005391522587423793, "get_robot_state": 0.0044250919529325895, "sim_render-ego0": 0.004487825610532933, "get_duckie_state": 1.811857987435905e-06, "in-drivable-lane": 4.20000000000006, "deviation-heading": 2.7695804307156413, "agent_compute-ego0": 0.014314406293923231, "complete-iteration": 0.1915794897449109, "set_robot_commands": 0.0026084477279229375, "deviation-center-line": 1.2080932275579792, "driven_lanedir_consec": 2.1675671005500683, "sim_compute_sim_state": 0.011624163743445423, "sim_compute_performance-ego0": 0.0024499357208725096}, "LF-norm-zigzag-000-ego0": {"driven_any": 2.092370889449156, "get_ui_image": 0.03964996121539827, "step_physics": 0.1523399889577322, "survival_time": 27.500000000000256, "driven_lanedir": 1.2402336142808588, "get_state_dump": 0.0052462307814028645, "get_robot_state": 0.004366149054248623, "sim_render-ego0": 0.0045839792587363355, "get_duckie_state": 1.7407585185582322e-06, "in-drivable-lane": 12.400000000000176, "deviation-heading": 2.3838113739399076, "agent_compute-ego0": 0.01489896306974269, "complete-iteration": 0.2390783045123147, "set_robot_commands": 0.002603166981747276, "deviation-center-line": 1.3772615134231143, "driven_lanedir_consec": 1.2402336142808588, "sim_compute_sim_state": 0.012801131838679095, "sim_compute_performance-ego0": 0.002471892240908531}, "LF-norm-techtrack-000-ego0": {"driven_any": 7.063567132962424, "get_ui_image": 0.033380803815728916, "step_physics": 0.11198030701287084, "survival_time": 59.99999999999873, "driven_lanedir": 6.8538346452510535, "get_state_dump": 0.004838983978855918, "get_robot_state": 0.0038388793811909264, "sim_render-ego0": 0.0040613953815113995, "get_duckie_state": 1.474383669431561e-06, "in-drivable-lane": 0.0, "deviation-heading": 12.686158690992796, "agent_compute-ego0": 0.013270107931538882, "complete-iteration": 0.1891673957179925, "set_robot_commands": 0.0022921445069960214, "deviation-center-line": 5.192180066600432, "driven_lanedir_consec": 6.8538346452510535, "sim_compute_sim_state": 0.01327484295231218, "sim_compute_performance-ego0": 0.002132496766305585}, "LF-norm-small_loop-000-ego0": {"driven_any": 6.507539808419359, "get_ui_image": 0.026466875250988654, "step_physics": 0.09419848460341176, "survival_time": 59.99999999999873, "driven_lanedir": 5.007103401547073, "get_state_dump": 0.004882751952400811, "get_robot_state": 0.0037659848361686304, "sim_render-ego0": 0.004059473739674844, "get_duckie_state": 1.3380026837173448e-06, "in-drivable-lane": 18.749999999999044, "deviation-heading": 9.626710502089995, "agent_compute-ego0": 0.012828406644403488, "complete-iteration": 0.15689777494171675, "set_robot_commands": 0.0023330050046795315, "deviation-center-line": 2.279206784090598, "driven_lanedir_consec": 5.007103401547073, "sim_compute_sim_state": 0.006201842345365576, "sim_compute_performance-ego0": 0.0020690899308178448}}
set_robot_commands_max0.0026084477279229375
set_robot_commands_mean0.0024591910553364415
set_robot_commands_median0.002468085993213404
set_robot_commands_min0.0022921445069960214
sim_compute_performance-ego0_max0.002471892240908531
sim_compute_performance-ego0_mean0.0022808536647261177
sim_compute_performance-ego0_median0.0022912162435890474
sim_compute_performance-ego0_min0.0020690899308178448
sim_compute_sim_state_max0.01327484295231218
sim_compute_sim_state_mean0.01097549521995057
sim_compute_sim_state_median0.01221264779106226
sim_compute_sim_state_min0.006201842345365576
sim_render-ego0_max0.0045839792587363355
sim_render-ego0_mean0.004298168497613878
sim_render-ego0_median0.004274610496022166
sim_render-ego0_min0.004059473739674844
simulation-passed1
step_physics_max0.1523399889577322
step_physics_mean0.11838988976792893
step_physics_median0.11351054275528584
step_physics_min0.09419848460341176
survival_time_max59.99999999999873
survival_time_mean41.69999999999946
survival_time_min19.30000000000014
No reset possible
57184LFv-simsuccessyes0:27:58
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
57182LFv-simsuccessyes0:26:12
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
57174LFv-simsuccessyes0:24:29
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
50941LFv-simerrorno0:08:55
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140709365494544
- M:video_aido:cmdline(in:/;out:/) 140709365494064
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40482LFv-simsuccessno0:11:30
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40481LFv-simsuccessno0:09:13
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
39767LFv-simsuccessno0:09:10
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38625LFv-simsuccessno0:08:48
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible