Duckietown Challenges Home Challenges Submissions

Submission 10723

Submission10723
Competingyes
Challengeaido5-LF-sim-validation
UserDishank Bansal 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 57936
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

57936

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
57936LFv-simsuccessyes0:34:49
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median6.19554939172707
survival_time_median59.99999999999873
deviation-center-line_median3.769356613898575
in-drivable-lane_median7.825000000000028


other stats
agent_compute-ego0_max0.01282561172752158
agent_compute-ego0_mean0.01266175890643829
agent_compute-ego0_median0.012654996831450832
agent_compute-ego0_min0.012511430235329912
complete-iteration_max0.2081845300977772
complete-iteration_mean0.18042153229225089
complete-iteration_median0.17862044375306066
complete-iteration_min0.15626071156510504
deviation-center-line_max4.6428389153842735
deviation-center-line_mean3.842389900132802
deviation-center-line_min3.188007457349784
deviation-heading_max11.53848338470762
deviation-heading_mean9.593146705049604
deviation-heading_median10.100217710962331
deviation-heading_min6.6336680135661386
driven_any_max7.921177251351979
driven_any_mean7.919370806707661
driven_any_median7.920972172760981
driven_any_min7.914361629956705
driven_lanedir_consec_max7.114847946805265
driven_lanedir_consec_mean5.817277716602344
driven_lanedir_consec_min3.76316413614997
driven_lanedir_max7.114847946805265
driven_lanedir_mean6.639771436884703
driven_lanedir_median6.658422643258609
driven_lanedir_min6.127392514216329
get_duckie_state_max1.4100642525881752e-06
get_duckie_state_mean1.323659850794707e-06
get_duckie_state_median1.3413774679344362e-06
get_duckie_state_min1.2018202147217813e-06
get_robot_state_max0.003701877633697485
get_robot_state_mean0.003619742979117973
get_robot_state_median0.0036173957670658057
get_robot_state_min0.003542302748642794
get_state_dump_max0.004625128865936813
get_state_dump_mean0.004565021874604872
get_state_dump_median0.004600537706672103
get_state_dump_min0.00443388321913847
get_ui_image_max0.03545147413814395
get_ui_image_mean0.03004430225747114
get_ui_image_median0.029460342599390742
get_ui_image_min0.02580504969295911
in-drivable-lane_max11.049999999999851
in-drivable-lane_mean7.624999999999967
in-drivable-lane_min3.7999999999999625
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 7.921177251351979, "get_ui_image": 0.02807949524338696, "step_physics": 0.09959948370597642, "survival_time": 59.99999999999873, "driven_lanedir": 6.917500087090627, "get_state_dump": 0.004603345030848927, "get_robot_state": 0.003661194411443731, "sim_render-ego0": 0.003845166802704086, "get_duckie_state": 1.362817273548898e-06, "in-drivable-lane": 6.250000000000089, "deviation-heading": 6.6336680135661386, "agent_compute-ego0": 0.0126870365365161, "complete-iteration": 0.1666453039517113, "set_robot_commands": 0.0022050792430461596, "deviation-center-line": 3.539691150194742, "driven_lanedir_consec": 3.76316413614997, "sim_compute_sim_state": 0.009885220206051842, "sim_compute_performance-ego0": 0.0019928533568370353}, "LF-norm-zigzag-000-ego0": {"driven_any": 7.920964777560345, "get_ui_image": 0.03545147413814395, "step_physics": 0.13083001239214412, "survival_time": 59.99999999999873, "driven_lanedir": 7.114847946805265, "get_state_dump": 0.00459773038249528, "get_robot_state": 0.003701877633697485, "sim_render-ego0": 0.003939275638348455, "get_duckie_state": 1.4100642525881752e-06, "in-drivable-lane": 3.7999999999999625, "deviation-heading": 11.53848338470762, "agent_compute-ego0": 0.01282561172752158, "complete-iteration": 0.2081845300977772, "set_robot_commands": 0.002258304751584373, "deviation-center-line": 3.999022077602408, "driven_lanedir_consec": 7.114847946805265, "sim_compute_sim_state": 0.012411155073370764, "sim_compute_performance-ego0": 0.00208057372595845}, "LF-norm-techtrack-000-ego0": {"driven_any": 7.914361629956705, "get_ui_image": 0.030841189955394532, "step_physics": 0.1187683634714322, "survival_time": 59.99999999999873, "driven_lanedir": 6.127392514216329, "get_state_dump": 0.00443388321913847, "get_robot_state": 0.003542302748642794, "sim_render-ego0": 0.0037664614748101153, "get_duckie_state": 1.2018202147217813e-06, "in-drivable-lane": 11.049999999999851, "deviation-heading": 11.117028605759206, "agent_compute-ego0": 0.012511430235329912, "complete-iteration": 0.19059558355441003, "set_robot_commands": 0.0021207074539349737, "deviation-center-line": 4.6428389153842735, "driven_lanedir_consec": 6.127392514216329, "sim_compute_sim_state": 0.012589344871133492, "sim_compute_performance-ego0": 0.0019400568429278296}, "LF-norm-small_loop-000-ego0": {"driven_any": 7.920979567961617, "get_ui_image": 0.02580504969295911, "step_physics": 0.0956897797135886, "survival_time": 59.99999999999873, "driven_lanedir": 6.39934519942659, "get_state_dump": 0.004625128865936813, "get_robot_state": 0.0035735971226878807, "sim_render-ego0": 0.003745836183292284, "get_duckie_state": 1.3199376623199742e-06, "in-drivable-lane": 9.399999999999968, "deviation-heading": 9.083406816165455, "agent_compute-ego0": 0.012622957126385564, "complete-iteration": 0.15626071156510504, "set_robot_commands": 0.0021496787853383897, "deviation-center-line": 3.188007457349784, "driven_lanedir_consec": 6.2637062692378125, "sim_compute_sim_state": 0.006052046989421861, "sim_compute_performance-ego0": 0.0019133360955637755}}
set_robot_commands_max0.002258304751584373
set_robot_commands_mean0.002183442558475974
set_robot_commands_median0.002177379014192274
set_robot_commands_min0.0021207074539349737
sim_compute_performance-ego0_max0.00208057372595845
sim_compute_performance-ego0_mean0.001981705005321773
sim_compute_performance-ego0_median0.0019664550998824324
sim_compute_performance-ego0_min0.0019133360955637755
sim_compute_sim_state_max0.012589344871133492
sim_compute_sim_state_mean0.01023444178499449
sim_compute_sim_state_median0.011148187639711302
sim_compute_sim_state_min0.006052046989421861
sim_render-ego0_max0.003939275638348455
sim_render-ego0_mean0.003824185024788735
sim_render-ego0_median0.0038058141387571007
sim_render-ego0_min0.003745836183292284
simulation-passed1
step_physics_max0.13083001239214412
step_physics_mean0.11122190982078534
step_physics_median0.1091839235887043
step_physics_min0.0956897797135886
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
51966LFv-simerrorno0:08:49
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140251513389600
- M:video_aido:cmdline(in:/;out:/) 140251513315088
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51952LFv-simerrorno0:10:14
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139934842578976
- M:video_aido:cmdline(in:/;out:/) 139934842526880
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51946LFv-simerrorno0:08:05
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139973442186784
- M:video_aido:cmdline(in:/;out:/) 139973442201296
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51840LFv-simhost-errorno0:07:47
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51836LFv-simerrorno0:09:34
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140317458153872
- M:video_aido:cmdline(in:/;out:/) 140317457648464
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51831LFv-simhost-errorno0:08:43
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
41038LFv-simsuccessno0:08:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
36635LFv-simsuccessno0:18:06
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible