Duckietown Challenges Home Challenges Submissions

Submission 10732

Submission10732
Competingyes
Challengeaido5-LF-sim-validation
UserYishu Malhotra 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 57944
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

57944

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
57944LFv-simsuccessyes0:28:11
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median2.440246919777531
survival_time_median59.22499999999877
deviation-center-line_median0.820258442015142
in-drivable-lane_median21.174999999999535


other stats
agent_compute-ego0_max0.012525652005122258
agent_compute-ego0_mean0.011881038322771638
agent_compute-ego0_median0.011757969558487128
agent_compute-ego0_min0.011482562168990032
complete-iteration_max0.20515633809684528
complete-iteration_mean0.17158842505101804
complete-iteration_median0.16769401207569973
complete-iteration_min0.14580933795582743
deviation-center-line_max3.177152237212606
deviation-center-line_mean1.2534527684174614
deviation-center-line_min0.1961419524269555
deviation-heading_max10.546493352350012
deviation-heading_mean5.096369593859241
deviation-heading_median3.993714087795544
deviation-heading_min1.851556847495864
driven_any_max8.586950522132026
driven_any_mean5.898475828420287
driven_any_median7.347598057281774
driven_any_min0.3117566769855757
driven_lanedir_consec_max7.650276553092979
driven_lanedir_consec_mean3.1676882695182957
driven_lanedir_consec_min0.13998268542514225
driven_lanedir_max7.650276553092979
driven_lanedir_mean3.1705784492808
driven_lanedir_median2.4460272793025397
driven_lanedir_min0.13998268542514225
get_duckie_state_max1.4608741825462408e-06
get_duckie_state_mean1.3369500213919174e-06
get_duckie_state_median1.3131880938857918e-06
get_duckie_state_min1.2605497152498454e-06
get_robot_state_max0.0037675712862585346
get_robot_state_mean0.0035952704195388262
get_robot_state_median0.003547119856079254
get_robot_state_min0.003519270679738262
get_state_dump_max0.004788518155741895
get_state_dump_mean0.0045752641654372836
get_state_dump_median0.004518785246405177
get_state_dump_min0.004474968013196888
get_ui_image_max0.033507689391032304
get_ui_image_mean0.029529898010586184
get_ui_image_median0.029768556592401955
get_ui_image_min0.02507478946650852
in-drivable-lane_max43.79999999999886
in-drivable-lane_mean21.537499999999486
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 8.586950522132026, "get_ui_image": 0.027492170826183767, "step_physics": 0.09108898979142543, "survival_time": 59.99999999999873, "driven_lanedir": 3.095258466603651, "get_state_dump": 0.0045285637829325575, "get_robot_state": 0.003558657349992255, "sim_render-ego0": 0.0038043815428569455, "get_duckie_state": 1.3513033038670571e-06, "in-drivable-lane": 40.04999999999908, "deviation-heading": 3.5480843506100013, "agent_compute-ego0": 0.011710673942851782, "complete-iteration": 0.15489119435230164, "set_robot_commands": 0.002178065286488656, "deviation-center-line": 0.8335164198644055, "driven_lanedir_consec": 3.095258466603651, "sim_compute_sim_state": 0.008519399176033013, "sim_compute_performance-ego0": 0.0019248373601756224}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.3117566769855757, "get_ui_image": 0.033507689391032304, "step_physics": 0.1346336331697974, "survival_time": 4.99999999999999, "driven_lanedir": 0.13998268542514225, "get_state_dump": 0.004474968013196888, "get_robot_state": 0.003519270679738262, "sim_render-ego0": 0.003704375559740727, "get_duckie_state": 1.2605497152498454e-06, "in-drivable-lane": 2.299999999999992, "deviation-heading": 1.851556847495864, "agent_compute-ego0": 0.011482562168990032, "complete-iteration": 0.20515633809684528, "set_robot_commands": 0.002092798157493667, "deviation-center-line": 0.1961419524269555, "driven_lanedir_consec": 0.13998268542514225, "sim_compute_sim_state": 0.00978411070191034, "sim_compute_performance-ego0": 0.0018771166848664236}, "LF-norm-techtrack-000-ego0": {"driven_any": 6.8908496680235825, "get_ui_image": 0.03204494235862015, "step_physics": 0.10913212278969268, "survival_time": 58.449999999998816, "driven_lanedir": 1.796796092001428, "get_state_dump": 0.004788518155741895, "get_robot_state": 0.0037675712862585346, "sim_render-ego0": 0.004007715445298415, "get_duckie_state": 1.4608741825462408e-06, "in-drivable-lane": 43.79999999999886, "deviation-heading": 4.439343824981086, "agent_compute-ego0": 0.012525652005122258, "complete-iteration": 0.18049682979909784, "set_robot_commands": 0.002288068054068802, "deviation-center-line": 0.8070004641658785, "driven_lanedir_consec": 1.785235372951411, "sim_compute_sim_state": 0.009813016704005056, "sim_compute_performance-ego0": 0.0020369144586416393}, "LF-norm-small_loop-000-ego0": {"driven_any": 7.804346446539965, "get_ui_image": 0.02507478946650852, "step_physics": 0.08712925998297064, "survival_time": 59.99999999999873, "driven_lanedir": 7.650276553092979, "get_state_dump": 0.004509006709877796, "get_robot_state": 0.003535582362166253, "sim_render-ego0": 0.0037091236924450166, "get_duckie_state": 1.2750728839045262e-06, "in-drivable-lane": 0.0, "deviation-heading": 10.546493352350012, "agent_compute-ego0": 0.011805265174122476, "complete-iteration": 0.14580933795582743, "set_robot_commands": 0.0021382104745018395, "deviation-center-line": 3.177152237212606, "driven_lanedir_consec": 7.650276553092979, "sim_compute_sim_state": 0.005959861383747796, "sim_compute_performance-ego0": 0.0018689128183306109}}
set_robot_commands_max0.002288068054068802
set_robot_commands_mean0.002174285493138241
set_robot_commands_median0.002158137880495248
set_robot_commands_min0.002092798157493667
sim_compute_performance-ego0_max0.0020369144586416393
sim_compute_performance-ego0_mean0.001926945330503574
sim_compute_performance-ego0_median0.0019009770225210231
sim_compute_performance-ego0_min0.0018689128183306109
sim_compute_sim_state_max0.009813016704005056
sim_compute_sim_state_mean0.00851909699142405
sim_compute_sim_state_median0.009151754938971676
sim_compute_sim_state_min0.005959861383747796
sim_render-ego0_max0.004007715445298415
sim_render-ego0_mean0.003806399060085276
sim_render-ego0_median0.003756752617650981
sim_render-ego0_min0.003704375559740727
simulation-passed1
step_physics_max0.1346336331697974
step_physics_mean0.10549600143347154
step_physics_median0.10011055629055904
step_physics_min0.08712925998297064
survival_time_max59.99999999999873
survival_time_mean45.862499999999066
survival_time_min4.99999999999999
No reset possible
57943LFv-simsuccessyes0:21:17
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51823LFv-simerrorno0:11:33
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140269372849792
- M:video_aido:cmdline(in:/;out:/) 140269372850128
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51814LFv-simerrorno0:08:17
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140641381012912
- M:video_aido:cmdline(in:/;out:/) 140640927117856
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51809LFv-simerrorno0:08:27
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139994251419408
- M:video_aido:cmdline(in:/;out:/) 139994249294512
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51802LFv-simhost-errorno0:08:08
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
41029LFv-simsuccessno0:08:57
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
41028LFv-simsuccessno0:08:59
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
36660LFv-simsuccessno0:10:52
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible