Duckietown Challenges Home Challenges Submissions

Submission 10891

Submission10891
Competingyes
Challengeaido5-LF-sim-validation
UserAyman Shams 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 57578
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

57578

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
57578LFv-simsuccessyes0:06:50
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.12238546976774922
survival_time_median9.675000000000017
deviation-center-line_median0.0673671477991638
in-drivable-lane_median7.325000000000015


other stats
agent_compute-ego0_max0.014051736669337496
agent_compute-ego0_mean0.013410344224642365
agent_compute-ego0_median0.01362242781642244
agent_compute-ego0_min0.012344784596387078
complete-iteration_max0.3137652468174062
complete-iteration_mean0.2265565290581073
complete-iteration_median0.20392095341401945
complete-iteration_min0.18461896258698413
deviation-center-line_max0.4772903200959861
deviation-center-line_mean0.16851525484003935
deviation-center-line_min0.062036403665843794
deviation-heading_max0.8942134132303006
deviation-heading_mean0.7894764165640882
deviation-heading_median0.8323467346959377
deviation-heading_min0.5989987836341772
driven_any_max3.3143444588639857
driven_any_mean1.7970818159020785
driven_any_median1.8049219876359264
driven_any_min0.264138829472475
driven_lanedir_consec_max0.6262444591942045
driven_lanedir_consec_mean0.239015846701344
driven_lanedir_consec_min0.08504798807567315
driven_lanedir_max0.6262444591942045
driven_lanedir_mean0.239015846701344
driven_lanedir_median0.12238546976774922
driven_lanedir_min0.08504798807567315
get_duckie_state_max1.5278539714023206e-06
get_duckie_state_mean1.4661532784768124e-06
get_duckie_state_median1.4911693518895156e-06
get_duckie_state_min1.3544204387258976e-06
get_robot_state_max0.004280018383229273
get_robot_state_mean0.003858695607334625
get_robot_state_median0.0037405404257780366
get_robot_state_min0.003673683194553151
get_state_dump_max0.005131815312176767
get_state_dump_mean0.004960540122322898
get_state_dump_median0.004982583407106176
get_state_dump_min0.004745178362902473
get_ui_image_max0.03550969286167875
get_ui_image_mean0.03108958419335034
get_ui_image_median0.030202242828363375
get_ui_image_min0.028444158254995855
in-drivable-lane_max15.600000000000104
in-drivable-lane_mean7.862500000000034
in-drivable-lane_min1.1999999999999995
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 2.4233921896555337, "get_ui_image": 0.028738925579508304, "step_physics": 0.12096418222419827, "survival_time": 12.600000000000044, "driven_lanedir": 0.6262444591942045, "get_state_dump": 0.004913188723236204, "get_robot_state": 0.0037662860433103538, "sim_render-ego0": 0.004065805744276688, "get_duckie_state": 1.4606671842190588e-06, "in-drivable-lane": 9.150000000000048, "deviation-heading": 0.8341153889260199, "agent_compute-ego0": 0.013352615559996351, "complete-iteration": 0.18967509269714355, "set_robot_commands": 0.0022597633331660695, "deviation-center-line": 0.4772903200959861, "driven_lanedir_consec": 0.6262444591942045, "sim_compute_sim_state": 0.009460754545309796, "sim_compute_performance-ego0": 0.002061594145100107}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.264138829472475, "get_ui_image": 0.03550969286167875, "step_physics": 0.23728894172830783, "survival_time": 2.3, "driven_lanedir": 0.08504798807567315, "get_state_dump": 0.005051978090976147, "get_robot_state": 0.00371479480824572, "sim_render-ego0": 0.0040851400253620555, "get_duckie_state": 1.3544204387258976e-06, "in-drivable-lane": 1.1999999999999995, "deviation-heading": 0.8942134132303006, "agent_compute-ego0": 0.014051736669337496, "complete-iteration": 0.3137652468174062, "set_robot_commands": 0.0024503849922342504, "deviation-center-line": 0.062036403665843794, "driven_lanedir_consec": 0.08504798807567315, "sim_compute_sim_state": 0.009316434251501204, "sim_compute_performance-ego0": 0.0022061124761053857}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.1864517856163197, "get_ui_image": 0.03166556007721845, "step_physics": 0.15025430917739868, "survival_time": 6.749999999999984, "driven_lanedir": 0.09863855581393244, "get_state_dump": 0.004745178362902473, "get_robot_state": 0.003673683194553151, "sim_render-ego0": 0.003787524559918572, "get_duckie_state": 1.5216715195599723e-06, "in-drivable-lane": 5.499999999999983, "deviation-heading": 0.8305780804658554, "agent_compute-ego0": 0.012344784596387078, "complete-iteration": 0.21816681413089528, "set_robot_commands": 0.002193201990688548, "deviation-center-line": 0.06917297257188829, "driven_lanedir_consec": 0.09863855581393244, "sim_compute_sim_state": 0.007465651806663065, "sim_compute_performance-ego0": 0.0019453115323010612}, "LF-norm-small_loop-000-ego0": {"driven_any": 3.3143444588639857, "get_ui_image": 0.028444158254995855, "step_physics": 0.11777041579139304, "survival_time": 16.850000000000104, "driven_lanedir": 0.146132383721566, "get_state_dump": 0.005131815312176767, "get_robot_state": 0.004280018383229273, "sim_render-ego0": 0.004446559403775007, "get_duckie_state": 1.5278539714023206e-06, "in-drivable-lane": 15.600000000000104, "deviation-heading": 0.5989987836341772, "agent_compute-ego0": 0.01389224007284853, "complete-iteration": 0.18461896258698413, "set_robot_commands": 0.002528096091817822, "deviation-center-line": 0.06556132302643934, "driven_lanedir_consec": 0.146132383721566, "sim_compute_sim_state": 0.00571219949327277, "sim_compute_performance-ego0": 0.0023144666965191183}}
set_robot_commands_max0.002528096091817822
set_robot_commands_mean0.0023578616019766727
set_robot_commands_median0.00235507416270016
set_robot_commands_min0.002193201990688548
sim_compute_performance-ego0_max0.0023144666965191183
sim_compute_performance-ego0_mean0.002131871212506418
sim_compute_performance-ego0_median0.0021338533106027464
sim_compute_performance-ego0_min0.0019453115323010612
sim_compute_sim_state_max0.009460754545309796
sim_compute_sim_state_mean0.00798876002418671
sim_compute_sim_state_median0.008391043029082135
sim_compute_sim_state_min0.00571219949327277
sim_render-ego0_max0.004446559403775007
sim_render-ego0_mean0.00409625743333308
sim_render-ego0_median0.0040754728848193715
sim_render-ego0_min0.003787524559918572
simulation-passed1
step_physics_max0.23728894172830783
step_physics_mean0.15656946223032447
step_physics_median0.1356092457007985
step_physics_min0.11777041579139304
survival_time_max16.850000000000104
survival_time_mean9.625000000000034
survival_time_min2.3
No reset possible
57575LFv-simtimeoutyes----No reset possible
57574LFv-simtimeoutyes----No reset possible
57572LFv-simsuccessyes0:07:23
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
57571LFv-simsuccessyes0:10:50
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
57568LFv-simtimeoutyes----No reset possible
51340LFv-simerrorno0:03:31
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139953055153696
- M:video_aido:cmdline(in:/;out:/) 139953055152112
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51333LFv-simerrorno0:02:43
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140430162782096
- M:video_aido:cmdline(in:/;out:/) 140425850932912
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51319LFv-simerrorno0:03:42
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140414325525376
- M:video_aido:cmdline(in:/;out:/) 140414359732864
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40659LFv-simsuccessno0:11:34
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40648LFv-simsuccessno0:09:39
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38356LFv-simsuccessno0:19:46
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38355LFv-simsuccessno0:07:48
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible