Duckietown Challenges Home Challenges Submissions

Submission 10999

Submission10999
Competingyes
Challengeaido5-LF-sim-validation
UserAyman Shams 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 57176
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

57176

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
57176LFv-simsuccessyes0:24:32
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.1670193616572586
survival_time_median46.349999999999504
deviation-center-line_median1.3636645051576646
in-drivable-lane_median34.59999999999953


other stats
agent_compute-ego0_max0.02241141492381481
agent_compute-ego0_mean0.015822120837122613
agent_compute-ego0_median0.01386969683363157
agent_compute-ego0_min0.0131376747574125
complete-iteration_max0.27795348097296324
complete-iteration_mean0.20726083700943376
complete-iteration_median0.1880269285834917
complete-iteration_min0.17503600989778836
deviation-center-line_max1.7719608176586736
deviation-center-line_mean1.1504132347752587
deviation-center-line_min0.10236311112703228
deviation-heading_max5.928799529925811
deviation-heading_mean3.799735718941669
deviation-heading_median3.7242722883440846
deviation-heading_min1.8215987691526925
driven_any_max8.178564239835756
driven_any_mean5.164010570238434
driven_any_median6.112424733184762
driven_any_min0.252628574748456
driven_lanedir_consec_max1.4520349948729487
driven_lanedir_consec_mean0.9594248963769596
driven_lanedir_consec_min0.051625867320372
driven_lanedir_max1.4520349948729487
driven_lanedir_mean0.9594248963769596
driven_lanedir_median1.1670193616572586
driven_lanedir_min0.051625867320372
get_duckie_state_max1.6913624429186616e-06
get_duckie_state_mean1.5764816232660429e-06
get_duckie_state_median1.5853739836636711e-06
get_duckie_state_min1.4438160828181677e-06
get_robot_state_max0.004224366292072077
get_robot_state_mean0.003977411674999924
get_robot_state_median0.0039514502839130515
get_robot_state_min0.0037823798401015146
get_state_dump_max0.005158089479737039
get_state_dump_mean0.004898884274098832
get_state_dump_median0.00487986003213069
get_state_dump_min0.004677727552396911
get_ui_image_max0.03756184087080114
get_ui_image_mean0.03097777957884277
get_ui_image_median0.029479410215502697
get_ui_image_min0.027390457013564545
in-drivable-lane_max49.399999999998805
in-drivable-lane_mean30.024999999999466
in-drivable-lane_min1.4999999999999951
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 6.381475432842526, "get_ui_image": 0.02842146580417951, "step_physics": 0.10549110422531764, "survival_time": 47.94999999999941, "driven_lanedir": 1.4520349948729487, "get_state_dump": 0.004803993552923202, "get_robot_state": 0.003926443308591843, "sim_render-ego0": 0.0041011857489744825, "get_duckie_state": 1.5579164028167724e-06, "in-drivable-lane": 34.8499999999995, "deviation-heading": 4.619355598884623, "agent_compute-ego0": 0.01320394203066826, "complete-iteration": 0.17503600989778836, "set_robot_commands": 0.002331157028675079, "deviation-center-line": 1.7719608176586736, "driven_lanedir_consec": 1.4520349948729487, "sim_compute_sim_state": 0.010409267743428548, "sim_compute_performance-ego0": 0.0022444297870000203}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.252628574748456, "get_ui_image": 0.03756184087080114, "step_physics": 0.19660168184953575, "survival_time": 3.349999999999996, "driven_lanedir": 0.051625867320372, "get_state_dump": 0.004955726511338178, "get_robot_state": 0.00397645725923426, "sim_render-ego0": 0.00427029062719906, "get_duckie_state": 1.6128315645105698e-06, "in-drivable-lane": 1.4999999999999951, "deviation-heading": 1.8215987691526925, "agent_compute-ego0": 0.014535451636594884, "complete-iteration": 0.27795348097296324, "set_robot_commands": 0.0027575142243329216, "deviation-center-line": 0.10236311112703228, "driven_lanedir_consec": 0.051625867320372, "sim_compute_sim_state": 0.01072098227108226, "sim_compute_performance-ego0": 0.0024669415810528923}, "LF-norm-techtrack-000-ego0": {"driven_any": 5.843374033526997, "get_ui_image": 0.03053735462682588, "step_physics": 0.11493638397327492, "survival_time": 44.749999999999595, "driven_lanedir": 0.97876114220595, "get_state_dump": 0.004677727552396911, "get_robot_state": 0.0037823798401015146, "sim_render-ego0": 0.003957287541457585, "get_duckie_state": 1.4438160828181677e-06, "in-drivable-lane": 34.34999999999957, "deviation-heading": 5.928799529925811, "agent_compute-ego0": 0.0131376747574125, "complete-iteration": 0.1879671245281185, "set_robot_commands": 0.0022479423454829623, "deviation-center-line": 1.1567105188597897, "driven_lanedir_consec": 0.97876114220595, "sim_compute_sim_state": 0.012493272445031576, "sim_compute_performance-ego0": 0.0020998375756399973}, "LF-norm-small_loop-000-ego0": {"driven_any": 8.178564239835756, "get_ui_image": 0.027390457013564545, "step_physics": 0.11255497082782526, "survival_time": 59.99999999999873, "driven_lanedir": 1.355277581108567, "get_state_dump": 0.005158089479737039, "get_robot_state": 0.004224366292072077, "sim_render-ego0": 0.004286285641786955, "get_duckie_state": 1.6913624429186616e-06, "in-drivable-lane": 49.399999999998805, "deviation-heading": 2.8291889778035464, "agent_compute-ego0": 0.02241141492381481, "complete-iteration": 0.18808673263886488, "set_robot_commands": 0.0024957551646490675, "deviation-center-line": 1.5706184914555394, "driven_lanedir_consec": 1.355277581108567, "sim_compute_sim_state": 0.007091805500154392, "sim_compute_performance-ego0": 0.0023595022222183825}}
set_robot_commands_max0.0027575142243329216
set_robot_commands_mean0.0024580921907850074
set_robot_commands_median0.0024134560966620733
set_robot_commands_min0.0022479423454829623
sim_compute_performance-ego0_max0.0024669415810528923
sim_compute_performance-ego0_mean0.002292677791477823
sim_compute_performance-ego0_median0.0023019660046092014
sim_compute_performance-ego0_min0.0020998375756399973
sim_compute_sim_state_max0.012493272445031576
sim_compute_sim_state_mean0.010178831989924192
sim_compute_sim_state_median0.010565125007255404
sim_compute_sim_state_min0.007091805500154392
sim_render-ego0_max0.004286285641786955
sim_render-ego0_mean0.00415376238985452
sim_render-ego0_median0.0041857381880867715
sim_render-ego0_min0.003957287541457585
simulation-passed1
step_physics_max0.19660168184953575
step_physics_mean0.1323960352189884
step_physics_median0.11374567740055008
step_physics_min0.10549110422531764
survival_time_max59.99999999999873
survival_time_mean39.012499999999434
survival_time_min3.349999999999996
No reset possible
57171LFv-simsuccessyes0:24:00
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
57169LFv-simsuccessyes0:24:24
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
57163LFv-simsuccessyes0:25:57
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
50943LFv-simerrorno0:06:20
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140209668791984
- M:video_aido:cmdline(in:/;out:/) 140209668800224
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
50936LFv-simerrorno0:05:34
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140432271341456
- M:video_aido:cmdline(in:/;out:/) 140432271257904
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
50929LFv-simerrorno0:07:14
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140133795871856
- M:video_aido:cmdline(in:/;out:/) 140133796476432
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40476LFv-simsuccessno0:09:40
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40475LFv-simsuccessno0:11:42
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40474LFv-simsuccessno0:09:17
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
39766LFv-simsuccessno0:09:00
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38628LFv-simsuccessno0:11:52
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38627LFv-simsuccessno0:10:53
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible