Duckietown Challenges Home Challenges Submissions

Submission 10804

Submission10804
Competingyes
Challengeaido5-LF-sim-validation
UserMelisande Teng
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 57835
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

57835

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
57835LFv-simsuccessyes0:13:35
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.2721931486142222
survival_time_median17.125000000000107
deviation-center-line_median0.6399048451122546
in-drivable-lane_median7.200000000000083


other stats
agent_compute-ego0_max0.01336566224601698
agent_compute-ego0_mean0.012899559364437284
agent_compute-ego0_median0.012817724457464118
agent_compute-ego0_min0.012597126296803922
complete-iteration_max0.2160394285502059
complete-iteration_mean0.19324867682877375
complete-iteration_median0.1927596636663969
complete-iteration_min0.1714359514320953
deviation-center-line_max0.9046828451397072
deviation-center-line_mean0.5979551838050664
deviation-center-line_min0.20732819985604883
deviation-heading_max4.219221631839349
deviation-heading_mean3.2193884535078565
deviation-heading_median3.650072229361062
deviation-heading_min1.3581877234699538
driven_any_max5.125279736867736
driven_any_mean2.989754595355008
driven_any_median2.754755967488392
driven_any_min1.3242267095755116
driven_lanedir_consec_max1.9432554035940597
driven_lanedir_consec_mean1.2712217054671424
driven_lanedir_consec_min0.5972451210460661
driven_lanedir_max1.9432554035940597
driven_lanedir_mean1.2712217054671424
driven_lanedir_median1.2721931486142222
driven_lanedir_min0.5972451210460661
get_duckie_state_max1.801313415691859e-06
get_duckie_state_mean1.6428953786631475e-06
get_duckie_state_median1.646111903182959e-06
get_duckie_state_min1.478044292594813e-06
get_robot_state_max0.003952491357459785
get_robot_state_mean0.003871426359628183
get_robot_state_median0.0038652851739873857
get_robot_state_min0.003802643733078174
get_state_dump_max0.005052264802944587
get_state_dump_mean0.004857223605767701
get_state_dump_median0.0048378724156375105
get_state_dump_min0.004700884788851195
get_ui_image_max0.0350855736250288
get_ui_image_mean0.030756802512005983
get_ui_image_median0.03085220823684164
get_ui_image_min0.026237219949311847
in-drivable-lane_max21.65000000000025
in-drivable-lane_mean10.212500000000103
in-drivable-lane_min4.799999999999997
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 3.0216366327190616, "get_ui_image": 0.02823533073589808, "step_physics": 0.11096370766426354, "survival_time": 18.500000000000128, "driven_lanedir": 1.9432554035940597, "get_state_dump": 0.004957096917288644, "get_robot_state": 0.003883190874783498, "sim_render-ego0": 0.004049898157865211, "get_duckie_state": 1.801313415691859e-06, "in-drivable-lane": 6.3000000000000895, "deviation-heading": 3.4098986951108867, "agent_compute-ego0": 0.012848213997812606, "complete-iteration": 0.17998357860225872, "set_robot_commands": 0.002349314985249563, "deviation-center-line": 0.7173318222540908, "driven_lanedir_consec": 1.9432554035940597, "sim_compute_sim_state": 0.010381245548834376, "sim_compute_performance-ego0": 0.0022079938184218907}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.3242267095755116, "get_ui_image": 0.0350855736250288, "step_physics": 0.1406665893083208, "survival_time": 8.84999999999999, "driven_lanedir": 0.5972451210460661, "get_state_dump": 0.004718647913986377, "get_robot_state": 0.003802643733078174, "sim_render-ego0": 0.0040651425886689945, "get_duckie_state": 1.648838600415862e-06, "in-drivable-lane": 4.799999999999997, "deviation-heading": 1.3581877234699538, "agent_compute-ego0": 0.01278723491711563, "complete-iteration": 0.2160394285502059, "set_robot_commands": 0.0023311416754561863, "deviation-center-line": 0.20732819985604883, "driven_lanedir_consec": 0.5972451210460661, "sim_compute_sim_state": 0.010303525442487738, "sim_compute_performance-ego0": 0.0021773054358664523}, "LF-norm-techtrack-000-ego0": {"driven_any": 5.125279736867736, "get_ui_image": 0.0334690857377852, "step_physics": 0.12911066542501037, "survival_time": 32.15000000000031, "driven_lanedir": 1.5293166722388631, "get_state_dump": 0.005052264802944587, "get_robot_state": 0.003952491357459785, "sim_render-ego0": 0.00417774730587598, "get_duckie_state": 1.643385205950056e-06, "in-drivable-lane": 21.65000000000025, "deviation-heading": 4.219221631839349, "agent_compute-ego0": 0.01336566224601698, "complete-iteration": 0.2055357487305351, "set_robot_commands": 0.0023947887539123155, "deviation-center-line": 0.9046828451397072, "driven_lanedir_consec": 1.5293166722388631, "sim_compute_sim_state": 0.011710062900685375, "sim_compute_performance-ego0": 0.0022017848417625664}, "LF-norm-small_loop-000-ego0": {"driven_any": 2.4878753022577222, "get_ui_image": 0.026237219949311847, "step_physics": 0.10962354835075667, "survival_time": 15.750000000000089, "driven_lanedir": 1.0150696249895812, "get_state_dump": 0.004700884788851195, "get_robot_state": 0.0038473794731912737, "sim_render-ego0": 0.003953878638110584, "get_duckie_state": 1.478044292594813e-06, "in-drivable-lane": 8.100000000000076, "deviation-heading": 3.890245763611237, "agent_compute-ego0": 0.012597126296803922, "complete-iteration": 0.1714359514320953, "set_robot_commands": 0.002264603783812704, "deviation-center-line": 0.5624778679704187, "driven_lanedir_consec": 1.0150696249895812, "sim_compute_sim_state": 0.006044455721408506, "sim_compute_performance-ego0": 0.002071881596046158}}
set_robot_commands_max0.0023947887539123155
set_robot_commands_mean0.0023349622996076923
set_robot_commands_median0.0023402283303528746
set_robot_commands_min0.002264603783812704
sim_compute_performance-ego0_max0.0022079938184218907
sim_compute_performance-ego0_mean0.002164741423024267
sim_compute_performance-ego0_median0.0021895451388145096
sim_compute_performance-ego0_min0.002071881596046158
sim_compute_sim_state_max0.011710062900685375
sim_compute_sim_state_mean0.009609822403354
sim_compute_sim_state_median0.010342385495661056
sim_compute_sim_state_min0.006044455721408506
sim_render-ego0_max0.00417774730587598
sim_render-ego0_mean0.004061666672630192
sim_render-ego0_median0.004057520373267103
sim_render-ego0_min0.003953878638110584
simulation-passed1
step_physics_max0.1406665893083208
step_physics_mean0.12259112768708784
step_physics_median0.12003718654463694
step_physics_min0.10962354835075667
survival_time_max32.15000000000031
survival_time_mean18.812500000000128
survival_time_min8.84999999999999
No reset possible
57832LFv-simsuccessyes0:12:55
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
57831LFv-simsuccessyes0:10:42
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
57830LFv-simsuccessyes0:09:13
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51642LFv-simerrorno0:09:24
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139767044460800
- M:video_aido:cmdline(in:/;out:/) 139767044882736
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51619LFv-simerrorno0:05:57
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140715714917088
- M:video_aido:cmdline(in:/;out:/) 140715714920016
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40931LFv-simsuccessno0:09:27
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
37975LFv-simsuccessno0:12:09
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
37973LFv-simsuccessno0:08:22
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible