Duckietown Challenges Home Challenges Submissions

Submission 11658

Submission11658
Competingyes
Challengeaido5-LF-sim-validation
UserPhilippe Reddy 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 54109
Next
User labelsim-exercise-2
Admin priority50
Blessingn/a
User priority50

54109

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
54109LFv-simsuccessyes0:30:58
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median7.854601239939741
survival_time_median59.99999999999873
deviation-center-line_median3.511094389955798
in-drivable-lane_median10.799999999999772


other stats
agent_compute-ego0_max0.012567479644190955
agent_compute-ego0_mean0.012344828041307664
agent_compute-ego0_median0.012316044094042676
agent_compute-ego0_min0.01217974433295435
complete-iteration_max0.3070879863656085
complete-iteration_mean0.2540908638904844
complete-iteration_median0.25097279495045505
complete-iteration_min0.207329879295419
deviation-center-line_max3.712682552624023
deviation-center-line_mean2.7801026534912094
deviation-center-line_min0.3855392814292213
deviation-heading_max17.267782118580126
deviation-heading_mean12.65317898134152
deviation-heading_median16.113991502242126
deviation-heading_min1.1169508023017076
driven_any_max19.800636488289346
driven_any_mean15.137259053992452
driven_any_median19.79189277881553
driven_any_min1.164614170049412
driven_lanedir_consec_max15.718176428681538
driven_lanedir_consec_mean8.131998101160852
driven_lanedir_consec_min1.1006134960823846
driven_lanedir_max15.718176428681538
driven_lanedir_mean10.97705615223823
driven_lanedir_median13.544717342094495
driven_lanedir_min1.1006134960823846
get_duckie_state_max1.4908605570797123e-06
get_duckie_state_mean1.4091374816027272e-06
get_duckie_state_median1.4135706620208946e-06
get_duckie_state_min1.3185480452894073e-06
get_robot_state_max0.0040648980104953025
get_robot_state_mean0.0039071873954060286
get_robot_state_median0.003859669837539781
get_robot_state_min0.003844511896049252
get_state_dump_max0.005139975027676724
get_state_dump_mean0.004904814713188461
get_state_dump_median0.0048695554736901455
get_state_dump_min0.004740172877696829
get_ui_image_max0.03620797654856806
get_ui_image_mean0.03206889860161277
get_ui_image_median0.032388831951735315
get_ui_image_min0.027289953954412378
in-drivable-lane_max15.599999999999614
in-drivable-lane_mean9.29999999999979
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 19.800636488289346, "get_ui_image": 0.029809486856071477, "step_physics": 0.1674480928568717, "survival_time": 59.99999999999873, "driven_lanedir": 15.718176428681538, "get_state_dump": 0.0047785169774547005, "get_robot_state": 0.003873743879904258, "sim_render-ego0": 0.00391795553831534, "get_duckie_state": 1.3185480452894073e-06, "in-drivable-lane": 8.299999999999862, "deviation-heading": 17.267782118580126, "agent_compute-ego0": 0.01217974433295435, "complete-iteration": 0.23620490309201508, "set_robot_commands": 0.0023473776151099668, "deviation-center-line": 3.633128688052904, "driven_lanedir_consec": 15.718176428681538, "sim_compute_sim_state": 0.00961249178394886, "sim_compute_performance-ego0": 0.0021410443006606027}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.164614170049412, "get_ui_image": 0.03620797654856806, "step_physics": 0.23154811755470608, "survival_time": 4.549999999999992, "driven_lanedir": 1.1006134960823846, "get_state_dump": 0.0049605939699255905, "get_robot_state": 0.003845595795175303, "sim_render-ego0": 0.003925997277964716, "get_duckie_state": 1.4460605123768682e-06, "in-drivable-lane": 0.0, "deviation-heading": 1.1169508023017076, "agent_compute-ego0": 0.012381872405176577, "complete-iteration": 0.3070879863656085, "set_robot_commands": 0.0023211733154628587, "deviation-center-line": 0.3855392814292213, "driven_lanedir_consec": 1.1006134960823846, "sim_compute_sim_state": 0.009781210318855618, "sim_compute_performance-ego0": 0.002023023107777471}, "LF-norm-techtrack-000-ego0": {"driven_any": 19.783622133294244, "get_ui_image": 0.03496817704739916, "step_physics": 0.1867645328785359, "survival_time": 59.99999999999873, "driven_lanedir": 13.903695218781916, "get_state_dump": 0.005139975027676724, "get_robot_state": 0.0040648980104953025, "sim_render-ego0": 0.004109933513288791, "get_duckie_state": 1.4908605570797123e-06, "in-drivable-lane": 13.29999999999968, "deviation-heading": 16.281396530488955, "agent_compute-ego0": 0.012567479644190955, "complete-iteration": 0.265740686808895, "set_robot_commands": 0.002446309215917278, "deviation-center-line": 3.712682552624023, "driven_lanedir_consec": 9.61536637441263, "sim_compute_sim_state": 0.013289941141349291, "sim_compute_performance-ego0": 0.0022855809487272163}, "LF-norm-small_loop-000-ego0": {"driven_any": 19.800163424336812, "get_ui_image": 0.027289953954412378, "step_physics": 0.14453109614954304, "survival_time": 59.99999999999873, "driven_lanedir": 13.18573946540708, "get_state_dump": 0.004740172877696829, "get_robot_state": 0.003844511896049252, "sim_render-ego0": 0.0038649954466299648, "get_duckie_state": 1.3810808116649212e-06, "in-drivable-lane": 15.599999999999614, "deviation-heading": 15.946586473995294, "agent_compute-ego0": 0.012250215782908774, "complete-iteration": 0.207329879295419, "set_robot_commands": 0.002306990579800443, "deviation-center-line": 3.38906009185869, "driven_lanedir_consec": 6.093836105466851, "sim_compute_sim_state": 0.006387666302060803, "sim_compute_performance-ego0": 0.0020196251229977827}}
set_robot_commands_max0.002446309215917278
set_robot_commands_mean0.002355462681572637
set_robot_commands_median0.002334275465286413
set_robot_commands_min0.002306990579800443
sim_compute_performance-ego0_max0.0022855809487272163
sim_compute_performance-ego0_mean0.0021173183700407684
sim_compute_performance-ego0_median0.002082033704219037
sim_compute_performance-ego0_min0.0020196251229977827
sim_compute_sim_state_max0.013289941141349291
sim_compute_sim_state_mean0.00976782738655364
sim_compute_sim_state_median0.00969685105140224
sim_compute_sim_state_min0.006387666302060803
sim_render-ego0_max0.004109933513288791
sim_render-ego0_mean0.0039547204440497025
sim_render-ego0_median0.003921976408140028
sim_render-ego0_min0.0038649954466299648
simulation-passed1
step_physics_max0.23154811755470608
step_physics_mean0.18257295985991415
step_physics_median0.1771063128677038
step_physics_min0.14453109614954304
survival_time_max59.99999999999873
survival_time_mean46.13749999999904
survival_time_min4.549999999999992
No reset possible
49200LFv-simerrorno0:07:51
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140234756466144
- M:video_aido:cmdline(in:/;out:/) 140234758003920
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49192LFv-simerrorno0:06:00
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140562728264992
- M:video_aido:cmdline(in:/;out:/) 140562642979520
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49172LFv-simerrorno0:04:29
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140508696488256
- M:video_aido:cmdline(in:/;out:/) 140508696489168
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43319LFv-simsuccessno0:08:01
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43318LFv-simsuccessno0:08:02
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43314LFv-simsuccessno0:08:38
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43312LFv-simsuccessno0:08:30
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43311LFv-simsuccessno0:08:44
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43309LFv-simhost-errorno0:00:37
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 687, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 888, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1388, in write_logs
    services2id: Dict[str, str] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 937, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible