Duckietown Challenges Home Challenges Submissions

Submission 10765

Submission10765
Competingyes
Challengeaido5-LF-sim-validation
UserMelisande Teng
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 57898
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

57898

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
57898LFv-simsuccessyes0:17:05
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.665977740756281
survival_time_median24.800000000000217
deviation-center-line_median0.2092804184980336
in-drivable-lane_median20.675000000000217


other stats
agent_compute-ego0_max0.01291124587473662
agent_compute-ego0_mean0.012372131112406252
agent_compute-ego0_median0.012334541988429374
agent_compute-ego0_min0.011908194598029642
complete-iteration_max0.2090711391050052
complete-iteration_mean0.1817882211861229
complete-iteration_median0.18408015260429533
complete-iteration_min0.14992144043089578
deviation-center-line_max1.7945871309220285
deviation-center-line_mean0.5946995256128055
deviation-center-line_min0.16565013453312663
deviation-heading_max10.19616040767916
deviation-heading_mean3.554698248694878
deviation-heading_median1.4285533556727852
deviation-heading_min1.1655258757547844
driven_any_max12.330723817175883
driven_any_mean5.809772836296048
driven_any_median4.798920662012966
driven_any_min1.3105262039823788
driven_lanedir_consec_max6.0162073800613785
driven_lanedir_consec_mean1.9973208269049745
driven_lanedir_consec_min0.6411204460459565
driven_lanedir_max6.0162073800613785
driven_lanedir_mean1.9973208269049745
driven_lanedir_median0.665977740756281
driven_lanedir_min0.6411204460459565
get_duckie_state_max1.2249186419058538e-06
get_duckie_state_mean1.1620850134776802e-06
get_duckie_state_median1.1482210001075251e-06
get_duckie_state_min1.126979411789817e-06
get_robot_state_max0.003603132043055551
get_robot_state_mean0.00353861329875787
get_robot_state_median0.0035306703206695288
get_robot_state_min0.003489980510636872
get_state_dump_max0.004446655511856079
get_state_dump_mean0.0044165958596298635
get_state_dump_median0.004419160303276763
get_state_dump_min0.004381407320109847
get_ui_image_max0.03614472875408098
get_ui_image_mean0.030826953052445
get_ui_image_median0.03051671522209357
get_ui_image_min0.026129653011511895
in-drivable-lane_max28.099999999998776
in-drivable-lane_mean18.3624999999998
in-drivable-lane_min3.999999999999986
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 12.330723817175883, "get_ui_image": 0.02816305053323433, "step_physics": 0.10537014396661128, "survival_time": 59.99999999999873, "driven_lanedir": 6.0162073800613785, "get_state_dump": 0.004426533137630364, "get_robot_state": 0.003603132043055551, "sim_render-ego0": 0.003712787914037903, "get_duckie_state": 1.126979411789817e-06, "in-drivable-lane": 28.099999999998776, "deviation-heading": 10.19616040767916, "agent_compute-ego0": 0.012532425164977877, "complete-iteration": 0.1714565291392813, "set_robot_commands": 0.002161133597038072, "deviation-center-line": 1.7945871309220285, "driven_lanedir_consec": 6.0162073800613785, "sim_compute_sim_state": 0.00949537466209596, "sim_compute_performance-ego0": 0.0019162697756320212}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.3105262039823788, "get_ui_image": 0.03614472875408098, "step_physics": 0.13570753265829646, "survival_time": 7.599999999999981, "driven_lanedir": 0.6411204460459565, "get_state_dump": 0.004381407320109847, "get_robot_state": 0.003489980510636872, "sim_render-ego0": 0.003634381138421351, "get_duckie_state": 1.1313195322074142e-06, "in-drivable-lane": 3.999999999999986, "deviation-heading": 1.1655258757547844, "agent_compute-ego0": 0.012136658811880871, "complete-iteration": 0.2090711391050052, "set_robot_commands": 0.002055629406100005, "deviation-center-line": 0.19726481878174665, "driven_lanedir_consec": 0.6411204460459565, "sim_compute_sim_state": 0.009604382359124477, "sim_compute_performance-ego0": 0.0018424863129659415}, "LF-norm-techtrack-000-ego0": {"driven_any": 5.26887794448245, "get_ui_image": 0.032870379910952804, "step_physics": 0.12451606556989143, "survival_time": 27.550000000000257, "driven_lanedir": 0.6417059180922563, "get_state_dump": 0.004446655511856079, "get_robot_state": 0.003550117862397346, "sim_render-ego0": 0.003704692574514859, "get_duckie_state": 1.2249186419058538e-06, "in-drivable-lane": 23.500000000000252, "deviation-heading": 1.4696480185792251, "agent_compute-ego0": 0.01291124587473662, "complete-iteration": 0.1967037760693094, "set_robot_commands": 0.0020748471868210945, "deviation-center-line": 0.16565013453312663, "driven_lanedir_consec": 0.6417059180922563, "sim_compute_sim_state": 0.010686456725217296, "sim_compute_performance-ego0": 0.001869867245356242}, "LF-norm-small_loop-000-ego0": {"driven_any": 4.328963379543483, "get_ui_image": 0.026129653011511895, "step_physics": 0.09116655291475322, "survival_time": 22.05000000000018, "driven_lanedir": 0.6902495634203056, "get_state_dump": 0.004411787468923163, "get_robot_state": 0.003511222778941711, "sim_render-ego0": 0.003606929498560289, "get_duckie_state": 1.1651224680076358e-06, "in-drivable-lane": 17.850000000000176, "deviation-heading": 1.3874586927663453, "agent_compute-ego0": 0.011908194598029642, "complete-iteration": 0.14992144043089578, "set_robot_commands": 0.002063151398395521, "deviation-center-line": 0.22129601821432057, "driven_lanedir_consec": 0.6902495634203056, "sim_compute_sim_state": 0.0052045848035165085, "sim_compute_performance-ego0": 0.0018461020284108987}}
set_robot_commands_max0.002161133597038072
set_robot_commands_mean0.0020886903970886734
set_robot_commands_median0.0020689992926083076
set_robot_commands_min0.002055629406100005
sim_compute_performance-ego0_max0.0019162697756320212
sim_compute_performance-ego0_mean0.001868681340591276
sim_compute_performance-ego0_median0.0018579846368835704
sim_compute_performance-ego0_min0.0018424863129659415
sim_compute_sim_state_max0.010686456725217296
sim_compute_sim_state_mean0.00874769963748856
sim_compute_sim_state_median0.00954987851061022
sim_compute_sim_state_min0.0052045848035165085
sim_render-ego0_max0.003712787914037903
sim_render-ego0_mean0.0036646977813836
sim_render-ego0_median0.003669536856468106
sim_render-ego0_min0.003606929498560289
simulation-passed1
step_physics_max0.13570753265829646
step_physics_mean0.1141900737773881
step_physics_median0.11494310476825136
step_physics_min0.09116655291475322
survival_time_max59.99999999999873
survival_time_mean29.299999999999788
survival_time_min7.599999999999981
No reset possible
51757LFv-simerrorno0:02:10
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140379871608784
- M:video_aido:cmdline(in:/;out:/) 140379870456224
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51737LFv-simerrorno0:03:59
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139813976576256
- M:video_aido:cmdline(in:/;out:/) 139813976487872
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51732LFv-simerrorno0:01:48
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140476165180048
- M:video_aido:cmdline(in:/;out:/) 140476165178368
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40983LFv-simsuccessno0:08:56
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40982LFv-simsuccessno0:08:38
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
37138LFv-simsuccessno0:09:50
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
37137LFv-simsuccessno0:10:18
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible