Duckietown Challenges Home Challenges Submissions

Submission 11520

Submission11520
Competingyes
Challengeaido5-LF-sim-validation
UserOlga (Ge Ya) Xu 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 54477
Next
User labelsim-exercise-2
Admin priority50
Blessingn/a
User priority50

54477

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
54477LFv-simsuccessyes0:27:43
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median3.144357258234821
survival_time_median56.899999999998904
deviation-center-line_median1.7568790814921813
in-drivable-lane_median28.599999999998992


other stats
agent_compute-ego0_max0.012275119308230284
agent_compute-ego0_mean0.012178875229745572
agent_compute-ego0_median0.012200671327613802
agent_compute-ego0_min0.012039038955524405
complete-iteration_max0.19575219051129217
complete-iteration_mean0.16902690614112392
complete-iteration_median0.16697148484287871
complete-iteration_min0.1464124643674461
deviation-center-line_max2.6806964460853653
deviation-center-line_mean1.7041029770693552
deviation-center-line_min0.6219572992076927
deviation-heading_max16.14512681517091
deviation-heading_mean9.432952703754667
deviation-heading_median9.403506144115465
deviation-heading_min2.7796717116168317
driven_any_max10.675518798476784
driven_any_mean7.950484719212339
driven_any_median8.675817291101799
driven_any_min3.77478549616897
driven_lanedir_consec_max4.968084037889436
driven_lanedir_consec_mean3.025962428972756
driven_lanedir_consec_min0.8470511615319443
driven_lanedir_max4.968084037889436
driven_lanedir_mean3.184088003023793
driven_lanedir_median3.1958600776159063
driven_lanedir_min1.3765478189739222
get_duckie_state_max1.2269584081506217e-06
get_duckie_state_mean1.1949875441570703e-06
get_duckie_state_median1.2123589937889208e-06
get_duckie_state_min1.1282737808998186e-06
get_robot_state_max0.0036816268838861937
get_robot_state_mean0.003661297535917854
get_robot_state_median0.003661457747045954
get_robot_state_min0.003640647765693314
get_state_dump_max0.004673117304920257
get_state_dump_mean0.004581015939984254
get_state_dump_median0.00458820907277234
get_state_dump_min0.004474528309472081
get_ui_image_max0.03602716626970099
get_ui_image_mean0.03052050596619304
get_ui_image_median0.030122980903593027
get_ui_image_min0.025808895787885113
in-drivable-lane_max43.74999999999909
in-drivable-lane_mean28.462499999999295
in-drivable-lane_min12.900000000000109
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 10.675518798476784, "get_ui_image": 0.02817743976509531, "step_physics": 0.09026586327087197, "survival_time": 59.349999999998765, "driven_lanedir": 4.542670787909658, "get_state_dump": 0.004474528309472081, "get_robot_state": 0.003671316586760961, "sim_render-ego0": 0.0037774256985596937, "get_duckie_state": 1.1282737808998186e-06, "in-drivable-lane": 36.749999999998835, "deviation-heading": 8.554312493922689, "agent_compute-ego0": 0.012189175344075416, "complete-iteration": 0.15681581705908984, "set_robot_commands": 0.0022300126576664473, "deviation-center-line": 2.000792011379182, "driven_lanedir_consec": 4.439665149147487, "sim_compute_sim_state": 0.010011183894443191, "sim_compute_performance-ego0": 0.0019432374523946333}, "LF-norm-zigzag-000-ego0": {"driven_any": 8.734829266255975, "get_ui_image": 0.03602716626970099, "step_physics": 0.11848978396756366, "survival_time": 59.99999999999873, "driven_lanedir": 1.3765478189739222, "get_state_dump": 0.004673117304920257, "get_robot_state": 0.003651598907330947, "sim_render-ego0": 0.003807591955230992, "get_duckie_state": 1.2236570537735482e-06, "in-drivable-lane": 43.74999999999909, "deviation-heading": 16.14512681517091, "agent_compute-ego0": 0.012275119308230284, "complete-iteration": 0.19575219051129217, "set_robot_commands": 0.002204548210823764, "deviation-center-line": 1.5129661516051804, "driven_lanedir_consec": 0.8470511615319443, "sim_compute_sim_state": 0.012572690509538863, "sim_compute_performance-ego0": 0.001969383320740915}, "LF-norm-techtrack-000-ego0": {"driven_any": 8.616805315947623, "get_ui_image": 0.03206852204209074, "step_physics": 0.10303674772245076, "survival_time": 54.44999999999904, "driven_lanedir": 4.968084037889436, "get_state_dump": 0.004653417736018469, "get_robot_state": 0.003640647765693314, "sim_render-ego0": 0.0038615721081374983, "get_duckie_state": 1.2010609338042932e-06, "in-drivable-lane": 20.449999999999147, "deviation-heading": 10.25269979430824, "agent_compute-ego0": 0.012212167311152189, "complete-iteration": 0.17712715262666756, "set_robot_commands": 0.002311611613002392, "deviation-center-line": 2.6806964460853653, "driven_lanedir_consec": 4.968084037889436, "sim_compute_sim_state": 0.013250318798450155, "sim_compute_performance-ego0": 0.002015516736091824}, "LF-norm-small_loop-000-ego0": {"driven_any": 3.77478549616897, "get_ui_image": 0.025808895787885113, "step_physics": 0.08591600694964009, "survival_time": 23.200000000000195, "driven_lanedir": 1.8490493673221549, "get_state_dump": 0.00452300040952621, "get_robot_state": 0.0036816268838861937, "sim_render-ego0": 0.003789708947622648, "get_duckie_state": 1.2269584081506217e-06, "in-drivable-lane": 12.900000000000109, "deviation-heading": 2.7796717116168317, "agent_compute-ego0": 0.012039038955524405, "complete-iteration": 0.1464124643674461, "set_robot_commands": 0.0023372234836701424, "deviation-center-line": 0.6219572992076927, "driven_lanedir_consec": 1.8490493673221549, "sim_compute_sim_state": 0.006269467774257865, "sim_compute_performance-ego0": 0.0019643968151461695}}
set_robot_commands_max0.0023372234836701424
set_robot_commands_mean0.0022708489912906865
set_robot_commands_median0.0022708121353344195
set_robot_commands_min0.002204548210823764
sim_compute_performance-ego0_max0.002015516736091824
sim_compute_performance-ego0_mean0.0019731335810933854
sim_compute_performance-ego0_median0.001966890067943542
sim_compute_performance-ego0_min0.0019432374523946333
sim_compute_sim_state_max0.013250318798450155
sim_compute_sim_state_mean0.01052591524417252
sim_compute_sim_state_median0.011291937201991028
sim_compute_sim_state_min0.006269467774257865
sim_render-ego0_max0.0038615721081374983
sim_render-ego0_mean0.003809074677387708
sim_render-ego0_median0.00379865045142682
sim_render-ego0_min0.0037774256985596937
simulation-passed1
step_physics_max0.11848978396756366
step_physics_mean0.09942710047763162
step_physics_median0.09665130549666136
step_physics_min0.08591600694964009
survival_time_max59.99999999999873
survival_time_mean49.24999999999918
survival_time_min23.200000000000195
No reset possible
54476LFv-simsuccessyes0:16:50
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49643LFv-simerrorno0:02:01
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139797542834768
- M:video_aido:cmdline(in:/;out:/) 139797542836592
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49640LFv-simhost-errorno0:02:13
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49634LFv-simerrorno0:03:05
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139979234016416
- M:video_aido:cmdline(in:/;out:/) 139979233434256
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42730LFv-simsuccessno0:11:40
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42729LFv-simsuccessno0:06:52
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42728LFv-simhost-errorno0:00:11
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 687, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 888, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1388, in write_logs
    services2id: Dict[str, str] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 937, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42727LFv-simhost-errorno0:00:11
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 687, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 888, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1388, in write_logs
    services2id: Dict[str, str] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 937, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42726LFv-simsuccessno0:06:50
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible