Duckietown Challenges Home Challenges Submissions

Submission 11715

Submission11715
Competingyes
Challengeaido5-LF-sim-validation
UserJean-Sébastien Grondin 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 53927
Next
User labelsim-exercise-2
Admin priority50
Blessingn/a
User priority50

53927

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
53927LFv-simsuccessyes0:42:53
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median10.297606876438792
survival_time_median59.99999999999873
deviation-center-line_median4.191002396471539
in-drivable-lane_median1.6999999999999569


other stats
agent_compute-ego0_max0.013385764168859225
agent_compute-ego0_mean0.013166439175903547
agent_compute-ego0_median0.013214092369778368
agent_compute-ego0_min0.012851807795198235
complete-iteration_max0.3049127666479741
complete-iteration_mean0.26427353842867896
complete-iteration_median0.2667887508621025
complete-iteration_min0.21860388534253683
deviation-center-line_max4.376500771208886
deviation-center-line_mean4.190752626487683
deviation-center-line_min4.004504941798769
deviation-heading_max27.599451919135586
deviation-heading_mean25.735912412952032
deviation-heading_median25.62203358297327
deviation-heading_min24.100130566726012
driven_any_max12.612311034162264
driven_any_mean12.217800496416528
driven_any_median12.221934905859444
driven_any_min11.815021139784966
driven_lanedir_consec_max11.080563427142948
driven_lanedir_consec_mean9.791556077556043
driven_lanedir_consec_min7.49044713020365
driven_lanedir_max11.080563427142948
driven_lanedir_mean10.193362548973973
driven_lanedir_median10.297606876438792
driven_lanedir_min9.097673015875362
get_duckie_state_max2.4169410495932753e-06
get_duckie_state_mean2.248499613816692e-06
get_duckie_state_median2.282743747784236e-06
get_duckie_state_min2.011569910105023e-06
get_robot_state_max0.004185425848091374
get_robot_state_mean0.004060254654816843
get_robot_state_median0.004098242268177194
get_robot_state_min0.0038591082348216088
get_state_dump_max0.005425659444905836
get_state_dump_mean0.005257100139827554
get_state_dump_median0.005219229552072847
get_state_dump_min0.005164282010258684
get_ui_image_max0.03844425779496701
get_ui_image_mean0.03295876784288913
get_ui_image_median0.03290611311954622
get_ui_image_min0.027578587337497074
in-drivable-lane_max5.749999999999911
in-drivable-lane_mean2.349999999999957
in-drivable-lane_min0.25000000000000355
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 11.815021139784966, "get_ui_image": 0.030461705396018557, "step_physics": 0.17891569340060295, "survival_time": 59.99999999999873, "driven_lanedir": 9.669999525629278, "get_state_dump": 0.005182435570112573, "get_robot_state": 0.004142468418308737, "sim_render-ego0": 0.0041623411329461575, "get_duckie_state": 2.364532635869035e-06, "in-drivable-lane": 2.899999999999906, "deviation-heading": 26.36762870162932, "agent_compute-ego0": 0.013254147385875948, "complete-iteration": 0.25117747293324594, "set_robot_commands": 0.0025283087302405667, "deviation-center-line": 4.376500771208886, "driven_lanedir_consec": 9.669999525629278, "sim_compute_sim_state": 0.010074229959048, "sim_compute_performance-ego0": 0.0023416756194001133}, "LF-norm-zigzag-000-ego0": {"driven_any": 12.021395293970311, "get_ui_image": 0.03844425779496701, "step_physics": 0.22208285113357684, "survival_time": 59.99999999999873, "driven_lanedir": 9.097673015875362, "get_state_dump": 0.005164282010258684, "get_robot_state": 0.004185425848091374, "sim_render-ego0": 0.004213189006745071, "get_duckie_state": 2.2009548596994366e-06, "in-drivable-lane": 5.749999999999911, "deviation-heading": 27.599451919135586, "agent_compute-ego0": 0.012851807795198235, "complete-iteration": 0.3049127666479741, "set_robot_commands": 0.0025010370989822527, "deviation-center-line": 4.004504941798769, "driven_lanedir_consec": 7.49044713020365, "sim_compute_sim_state": 0.012987232327361984, "sim_compute_performance-ego0": 0.0023751117903227413}, "LF-norm-techtrack-000-ego0": {"driven_any": 12.612311034162264, "get_ui_image": 0.035350520843073885, "step_physics": 0.2014521845770716, "survival_time": 59.99999999999873, "driven_lanedir": 11.080563427142948, "get_state_dump": 0.005256023534033122, "get_robot_state": 0.004054016118045651, "sim_render-ego0": 0.004140033610754466, "get_duckie_state": 2.4169410495932753e-06, "in-drivable-lane": 0.5000000000000071, "deviation-heading": 24.100130566726012, "agent_compute-ego0": 0.013385764168859225, "complete-iteration": 0.2824000287909591, "set_robot_commands": 0.0024518494205014296, "deviation-center-line": 4.182810739685304, "driven_lanedir_consec": 11.080563427142948, "sim_compute_sim_state": 0.01389242905959003, "sim_compute_performance-ego0": 0.0023018392694681314}, "LF-norm-small_loop-000-ego0": {"driven_any": 12.422474517748576, "get_ui_image": 0.027578587337497074, "step_physics": 0.15356672792807904, "survival_time": 59.99999999999873, "driven_lanedir": 10.925214227248306, "get_state_dump": 0.005425659444905836, "get_robot_state": 0.0038591082348216088, "sim_render-ego0": 0.003957886977755557, "get_duckie_state": 2.011569910105023e-06, "in-drivable-lane": 0.25000000000000355, "deviation-heading": 24.87643846431722, "agent_compute-ego0": 0.01317403735368079, "complete-iteration": 0.21860388534253683, "set_robot_commands": 0.002304583763103501, "deviation-center-line": 4.199194053257774, "driven_lanedir_consec": 10.925214227248306, "sim_compute_sim_state": 0.006585351434178793, "sim_compute_performance-ego0": 0.00205606068302253}}
set_robot_commands_max0.0025283087302405667
set_robot_commands_mean0.0024464447532069377
set_robot_commands_median0.002476443259741841
set_robot_commands_min0.002304583763103501
sim_compute_performance-ego0_max0.0023751117903227413
sim_compute_performance-ego0_mean0.002268671840553379
sim_compute_performance-ego0_median0.002321757444434122
sim_compute_performance-ego0_min0.00205606068302253
sim_compute_sim_state_max0.01389242905959003
sim_compute_sim_state_mean0.010884810695044702
sim_compute_sim_state_median0.011530731143204992
sim_compute_sim_state_min0.006585351434178793
sim_render-ego0_max0.004213189006745071
sim_render-ego0_mean0.0041183626820503125
sim_render-ego0_median0.004151187371850312
sim_render-ego0_min0.003957886977755557
simulation-passed1
step_physics_max0.22208285113357684
step_physics_mean0.1890043642598326
step_physics_median0.1901839389888373
step_physics_min0.15356672792807904
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
48831LFv-simerrorno0:10:20
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140605056274832
- M:video_aido:cmdline(in:/;out:/) 140605056404352
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43707LFv-simsuccessno0:08:07
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43706LFv-simsuccessno0:06:25
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43705LFv-simhost-errorno0:00:11
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 687, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 888, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1388, in write_logs
    services2id: Dict[str, str] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 937, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43704LFv-simhost-errorno0:00:12
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 687, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 888, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1388, in write_logs
    services2id: Dict[str, str] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 937, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43702LFv-simhost-errorno0:00:12
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 687, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 888, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1388, in write_logs
    services2id: Dict[str, str] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 937, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43682LFv-simsuccessno0:06:58
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43681LFv-simsuccessno0:11:58
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43680LFv-simsuccessno0:13:33
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible