Duckietown Challenges Home Challenges Submissions

Submission 11566

Submission11566
Competingyes
Challengeaido5-LF-sim-validation
UserOlga (Ge Ya) Xu 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 54359
Next
User labelsim-exercise-2
Admin priority50
Blessingn/a
User priority50

54359

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
54359LFv-simsuccessyes0:22:55
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.9796683576507668
survival_time_median41.94999999999946
deviation-center-line_median0.5187958739171133
in-drivable-lane_median25.37499999999986


other stats
agent_compute-ego0_max0.013727632390767906
agent_compute-ego0_mean0.01314036133992394
agent_compute-ego0_median0.013341302508623174
agent_compute-ego0_min0.0121512079516815
complete-iteration_max0.19076999418939977
complete-iteration_mean0.17288786131441533
complete-iteration_median0.1745086981447251
complete-iteration_min0.15176405477881133
deviation-center-line_max2.707422069466323
deviation-center-line_mean1.0005629585276246
deviation-center-line_min0.25723801680994873
deviation-heading_max6.965955703974146
deviation-heading_mean3.347099446157851
deviation-heading_median2.827499860807864
deviation-heading_min0.7674423590415304
driven_any_max14.075675988260532
driven_any_mean8.569310590982166
driven_any_median8.498283869245904
driven_any_min3.2049986371763186
driven_lanedir_consec_max7.010548806162326
driven_lanedir_consec_mean2.390586603405387
driven_lanedir_consec_min0.5924608921576864
driven_lanedir_max7.010548806162326
driven_lanedir_mean2.390586603405387
driven_lanedir_median0.9796683576507668
driven_lanedir_min0.5924608921576864
get_duckie_state_max1.546485438974018e-06
get_duckie_state_mean1.3840824063960736e-06
get_duckie_state_median1.3452485439481584e-06
get_duckie_state_min1.2993470987139596e-06
get_robot_state_max0.003980464875573654
get_robot_state_mean0.003814171024070912
get_robot_state_median0.003860477062203035
get_robot_state_min0.0035552650963039225
get_state_dump_max0.004908831482889259
get_state_dump_mean0.004747129067075222
get_state_dump_median0.0047979814757359365
get_state_dump_min0.0044837218339397545
get_ui_image_max0.03586651919584886
get_ui_image_mean0.030787991583337452
get_ui_image_median0.03018641827056858
get_ui_image_min0.026912610596363788
in-drivable-lane_max51.34999999999866
in-drivable-lane_mean28.899999999999626
in-drivable-lane_min13.500000000000115
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 14.075675988260532, "get_ui_image": 0.026912610596363788, "step_physics": 0.08736821217501194, "survival_time": 59.99999999999873, "driven_lanedir": 7.010548806162326, "get_state_dump": 0.0044837218339397545, "get_robot_state": 0.0035552650963039225, "sim_render-ego0": 0.0036743127932457207, "get_duckie_state": 1.3052474251396948e-06, "in-drivable-lane": 30.599999999999515, "deviation-heading": 6.965955703974146, "agent_compute-ego0": 0.0121512079516815, "complete-iteration": 0.15176405477881133, "set_robot_commands": 0.002105379382537664, "deviation-center-line": 2.707422069466323, "driven_lanedir_consec": 7.010548806162326, "sim_compute_sim_state": 0.009512369082829635, "sim_compute_performance-ego0": 0.0019209380947878517}, "LF-norm-zigzag-000-ego0": {"driven_any": 12.56903178426536, "get_ui_image": 0.03586651919584886, "step_physics": 0.10924778036233487, "survival_time": 59.99999999999873, "driven_lanedir": 1.3155083176706643, "get_state_dump": 0.0047929257179279315, "get_robot_state": 0.003839331602276017, "sim_render-ego0": 0.003984958107128032, "get_duckie_state": 1.3852496627566222e-06, "in-drivable-lane": 51.34999999999866, "deviation-heading": 3.956849500935315, "agent_compute-ego0": 0.013727632390767906, "complete-iteration": 0.19076999418939977, "set_robot_commands": 0.0023503535792392853, "deviation-center-line": 0.7773210726519036, "driven_lanedir_consec": 1.3155083176706643, "sim_compute_sim_state": 0.014693098401745392, "sim_compute_performance-ego0": 0.002176801529057714}, "LF-norm-techtrack-000-ego0": {"driven_any": 3.2049986371763186, "get_ui_image": 0.033333091134669784, "step_physics": 0.1131332562782703, "survival_time": 17.400000000000112, "driven_lanedir": 0.5924608921576864, "get_state_dump": 0.0048030372335439424, "get_robot_state": 0.0038816225221300535, "sim_render-ego0": 0.004100206588264181, "get_duckie_state": 1.2993470987139596e-06, "in-drivable-lane": 13.500000000000115, "deviation-heading": 0.7674423590415304, "agent_compute-ego0": 0.01308287727115489, "complete-iteration": 0.18726130884492975, "set_robot_commands": 0.00234670557060351, "deviation-center-line": 0.25723801680994873, "driven_lanedir_consec": 0.5924608921576864, "sim_compute_sim_state": 0.01036034682418692, "sim_compute_performance-ego0": 0.002132251816014506}, "LF-norm-small_loop-000-ego0": {"driven_any": 4.427535954226448, "get_ui_image": 0.02703974540646738, "step_physics": 0.0968393283995308, "survival_time": 23.900000000000205, "driven_lanedir": 0.6438283976308692, "get_state_dump": 0.004908831482889259, "get_robot_state": 0.003980464875573654, "sim_render-ego0": 0.004016871243279762, "get_duckie_state": 1.546485438974018e-06, "in-drivable-lane": 20.150000000000208, "deviation-heading": 1.6981502206804124, "agent_compute-ego0": 0.013599727746091457, "complete-iteration": 0.16175608744452044, "set_robot_commands": 0.00233148980986848, "deviation-center-line": 0.2602706751823231, "driven_lanedir_consec": 0.6438283976308692, "sim_compute_sim_state": 0.006819131230013853, "sim_compute_performance-ego0": 0.00213031081914404}}
set_robot_commands_max0.0023503535792392853
set_robot_commands_mean0.0022834820855622347
set_robot_commands_median0.002339097690235995
set_robot_commands_min0.002105379382537664
sim_compute_performance-ego0_max0.002176801529057714
sim_compute_performance-ego0_mean0.0020900755647510283
sim_compute_performance-ego0_median0.002131281317579273
sim_compute_performance-ego0_min0.0019209380947878517
sim_compute_sim_state_max0.014693098401745392
sim_compute_sim_state_mean0.01034623638469395
sim_compute_sim_state_median0.00993635795350828
sim_compute_sim_state_min0.006819131230013853
sim_render-ego0_max0.004100206588264181
sim_render-ego0_mean0.003944087182979424
sim_render-ego0_median0.0040009146752038975
sim_render-ego0_min0.0036743127932457207
simulation-passed1
step_physics_max0.1131332562782703
step_physics_mean0.10164714430378698
step_physics_median0.10304355438093282
step_physics_min0.08736821217501194
survival_time_max59.99999999999873
survival_time_mean40.32499999999944
survival_time_min17.400000000000112
No reset possible
54357LFv-simsuccessyes0:15:01
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49536LFv-simerrorno0:02:55
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139815374973440
- M:video_aido:cmdline(in:/;out:/) 139815374972336
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49519LFv-simerrorno0:07:45
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140552113870592
- M:video_aido:cmdline(in:/;out:/) 140552113428848
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42937LFv-simsuccessno0:07:11
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42936LFv-simsuccessno0:07:14
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42935LFv-simsuccessno0:08:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42934LFv-simhost-errorno0:00:12
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 687, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 888, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1388, in write_logs
    services2id: Dict[str, str] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 937, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42933LFv-simsuccessno0:07:10
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42932LFv-simsuccessno0:07:04
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42929LFv-simhost-errorno0:00:12
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 687, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 888, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1388, in write_logs
    services2id: Dict[str, str] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 937, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42928LFv-simsuccessno0:07:12
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42927LFv-simsuccessno0:08:32
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible