Duckietown Challenges Home Challenges Submissions

Submission 10881

Submission10881
Competingyes
Challengeaido5-LF-sim-validation
UserYishu Malhotra 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 57668
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

57668

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
57668LFv-simsuccessyes0:24:55
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median3.117472110889487
survival_time_median56.79999999999891
deviation-center-line_median1.4417941557499303
in-drivable-lane_median28.37499999999914


other stats
agent_compute-ego0_max0.014564788204500046
agent_compute-ego0_mean0.013559699818387987
agent_compute-ego0_median0.013578279613047552
agent_compute-ego0_min0.012517451842956795
complete-iteration_max0.2272700907170087
complete-iteration_mean0.1819904364912916
complete-iteration_median0.18010607625931296
complete-iteration_min0.14047950272953183
deviation-center-line_max1.998734616014289
deviation-center-line_mean1.2623666774422957
deviation-center-line_min0.1671437822550329
deviation-heading_max6.3737320523936924
deviation-heading_mean4.872601093714591
deviation-heading_median5.633575939830877
deviation-heading_min1.8495204428029184
driven_any_max7.501194740926674
driven_any_mean5.3789392098828746
driven_any_median6.857002448515324
driven_any_min0.30055720157417726
driven_lanedir_consec_max4.0087916025913035
driven_lanedir_consec_mean2.5952953171257462
driven_lanedir_consec_min0.13744544413270665
driven_lanedir_max4.0087916025913035
driven_lanedir_mean2.5952953171257462
driven_lanedir_median3.117472110889487
driven_lanedir_min0.13744544413270665
get_duckie_state_max1.593604892186772e-06
get_duckie_state_mean1.435420045617527e-06
get_duckie_state_median1.4465262974813523e-06
get_duckie_state_min1.2550226953206313e-06
get_robot_state_max0.004100479948629845
get_robot_state_mean0.003871730819792853
get_robot_state_median0.0038991539179815864
get_robot_state_min0.003588135494578391
get_state_dump_max0.005195140838623047
get_state_dump_mean0.004865763948030898
get_state_dump_median0.004912132951452344
get_state_dump_min0.004443649050595857
get_ui_image_max0.0347792937837798
get_ui_image_mean0.030468029022920208
get_ui_image_median0.03070742545435168
get_ui_image_min0.02567797139919767
in-drivable-lane_max36.14999999999925
in-drivable-lane_mean23.624999999999385
in-drivable-lane_min1.5999999999999943
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 6.896659122312882, "get_ui_image": 0.02834015590280211, "step_physics": 0.1015252895924052, "survival_time": 53.59999999999909, "driven_lanedir": 4.0087916025913035, "get_state_dump": 0.00489609928077776, "get_robot_state": 0.004011774107549975, "sim_render-ego0": 0.00410714087259625, "get_duckie_state": 1.593604892186772e-06, "in-drivable-lane": 26.049999999999834, "deviation-heading": 5.091280483746498, "agent_compute-ego0": 0.013164579590569004, "complete-iteration": 0.1723590583747942, "set_robot_commands": 0.002319269278065023, "deviation-center-line": 1.0553087567759611, "driven_lanedir_consec": 4.0087916025913035, "sim_compute_sim_state": 0.011685966001085778, "sim_compute_performance-ego0": 0.002209676056405927}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.30055720157417726, "get_ui_image": 0.0347792937837798, "step_physics": 0.15015004147058245, "survival_time": 4.299999999999993, "driven_lanedir": 0.13744544413270665, "get_state_dump": 0.005195140838623047, "get_robot_state": 0.003786533728413198, "sim_render-ego0": 0.003982963233158506, "get_duckie_state": 1.397626153353987e-06, "in-drivable-lane": 1.5999999999999943, "deviation-heading": 1.8495204428029184, "agent_compute-ego0": 0.014564788204500046, "complete-iteration": 0.2272700907170087, "set_robot_commands": 0.0022559056336852326, "deviation-center-line": 0.1671437822550329, "driven_lanedir_consec": 0.13744544413270665, "sim_compute_sim_state": 0.010297756085450622, "sim_compute_performance-ego0": 0.0021585650827692844}, "LF-norm-techtrack-000-ego0": {"driven_any": 6.817345774717767, "get_ui_image": 0.03307469500590125, "step_physics": 0.10980194057652, "survival_time": 59.99999999999873, "driven_lanedir": 2.8083142335681694, "get_state_dump": 0.004928166622126927, "get_robot_state": 0.004100479948629845, "sim_render-ego0": 0.00444430574390116, "get_duckie_state": 1.495426441608718e-06, "in-drivable-lane": 36.14999999999925, "deviation-heading": 6.3737320523936924, "agent_compute-ego0": 0.0139919796355261, "complete-iteration": 0.18785309414383175, "set_robot_commands": 0.002565892510966, "deviation-center-line": 1.8282795547238997, "driven_lanedir_consec": 2.8083142335681694, "sim_compute_sim_state": 0.012547357989588347, "sim_compute_performance-ego0": 0.0022964725684960817}, "LF-norm-small_loop-000-ego0": {"driven_any": 7.501194740926674, "get_ui_image": 0.02567797139919767, "step_physics": 0.0805846695895993, "survival_time": 59.99999999999873, "driven_lanedir": 3.426629988210805, "get_state_dump": 0.004443649050595857, "get_robot_state": 0.003588135494578391, "sim_render-ego0": 0.003798272786390573, "get_duckie_state": 1.2550226953206313e-06, "in-drivable-lane": 30.699999999998447, "deviation-heading": 6.175871395915255, "agent_compute-ego0": 0.012517451842956795, "complete-iteration": 0.14047950272953183, "set_robot_commands": 0.0021480062819837433, "deviation-center-line": 1.998734616014289, "driven_lanedir_consec": 3.426629988210805, "sim_compute_sim_state": 0.005717553068060958, "sim_compute_performance-ego0": 0.0019243212167071263}}
set_robot_commands_max0.002565892510966
set_robot_commands_mean0.002322268426175
set_robot_commands_median0.002287587455875128
set_robot_commands_min0.0021480062819837433
sim_compute_performance-ego0_max0.0022964725684960817
sim_compute_performance-ego0_mean0.0021472587310946045
sim_compute_performance-ego0_median0.0021841205695876057
sim_compute_performance-ego0_min0.0019243212167071263
sim_compute_sim_state_max0.012547357989588347
sim_compute_sim_state_mean0.010062158286046428
sim_compute_sim_state_median0.0109918610432682
sim_compute_sim_state_min0.005717553068060958
sim_render-ego0_max0.00444430574390116
sim_render-ego0_mean0.004083170659011622
sim_render-ego0_median0.004045052052877378
sim_render-ego0_min0.003798272786390573
simulation-passed1
step_physics_max0.15015004147058245
step_physics_mean0.11051548530727671
step_physics_median0.1056636150844626
step_physics_min0.0805846695895993
survival_time_max59.99999999999873
survival_time_mean44.474999999999135
survival_time_min4.299999999999993
No reset possible
51383LFv-simerrorno0:07:28
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139750891146880
- M:video_aido:cmdline(in:/;out:/) 139750891144528
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51368LFv-simerrorno0:10:05
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139628669008240
- M:video_aido:cmdline(in:/;out:/) 139628669008624
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51359LFv-simerrorno0:07:47
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140630439734096
- M:video_aido:cmdline(in:/;out:/) 140630439735104
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51357LFv-simerrorno0:04:21
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139715040996752
- M:video_aido:cmdline(in:/;out:/) 139715041048800
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40666LFv-simsuccessno0:08:46
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38323LFv-simsuccessno0:09:45
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38321LFv-simsuccessno0:09:51
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible