Duckietown Challenges Home Challenges Submissions

Submission 10973

Submission10973
Competingyes
Challengeaido5-LF-sim-validation
UserYishu Malhotra 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 57272
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

57272

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
57272LFv-simsuccessyes0:18:18
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.4210264677470112
survival_time_median25.424999999999876
deviation-center-line_median0.7538164843051756
in-drivable-lane_median17.649999999999913


other stats
agent_compute-ego0_max0.012845634755508676
agent_compute-ego0_mean0.012421568287007889
agent_compute-ego0_median0.01241828286317733
agent_compute-ego0_min0.012004072666168214
complete-iteration_max0.23075875759124756
complete-iteration_mean0.19902416632934183
complete-iteration_median0.19911927310557703
complete-iteration_min0.16709936151496577
deviation-center-line_max1.7857220375701073
deviation-center-line_mean0.8588159515006792
deviation-center-line_min0.14190879982225874
deviation-heading_max11.96763258431313
deviation-heading_mean5.009593331886197
deviation-heading_median3.5763655224223556
deviation-heading_min0.9180096983869492
driven_any_max11.752950382102384
driven_any_mean5.519259206205342
driven_any_median4.826417731540104
driven_any_min0.6712509796387744
driven_lanedir_consec_max3.650215997531058
driven_lanedir_consec_mean1.6943355729773304
driven_lanedir_consec_min0.2850733588842411
driven_lanedir_max4.505535089011568
driven_lanedir_mean2.054785141416629
driven_lanedir_median1.714266058885353
driven_lanedir_min0.2850733588842411
get_duckie_state_max1.4729033222061925e-06
get_duckie_state_mean1.373763155191146e-06
get_duckie_state_median1.3784018669339176e-06
get_duckie_state_min1.2653455646905573e-06
get_robot_state_max0.003834957575929758
get_robot_state_mean0.003700945146955344
get_robot_state_median0.0036961450127257936
get_robot_state_min0.003576532986440031
get_state_dump_max0.004664481674110033
get_state_dump_mean0.004610116206834356
get_state_dump_median0.0046406148726160605
get_state_dump_min0.00449475340799527
get_ui_image_max0.03563255786895752
get_ui_image_mean0.02987828138050313
get_ui_image_median0.029023202443877676
get_ui_image_min0.025834162765299647
in-drivable-lane_max36.04999999999909
in-drivable-lane_mean18.499999999999726
in-drivable-lane_min2.6499999999999906
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 7.686718515548966, "get_ui_image": 0.02749820512348257, "step_physics": 0.12369320017056704, "survival_time": 41.84999999999976, "driven_lanedir": 1.8761643609853844, "get_state_dump": 0.0046208140958180575, "get_robot_state": 0.0037637428338317142, "sim_render-ego0": 0.0038884835482212695, "get_duckie_state": 1.4729033222061925e-06, "in-drivable-lane": 32.64999999999982, "deviation-heading": 5.592217147777675, "agent_compute-ego0": 0.012728976463645625, "complete-iteration": 0.19118838606131103, "set_robot_commands": 0.002212159104449652, "deviation-center-line": 0.896768571848211, "driven_lanedir_consec": 1.2896851787087005, "sim_compute_sim_state": 0.010532061638297216, "sim_compute_performance-ego0": 0.002151022662979891}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.6712509796387744, "get_ui_image": 0.03563255786895752, "step_physics": 0.1566799259185791, "survival_time": 4.94999999999999, "driven_lanedir": 0.2850733588842411, "get_state_dump": 0.004660415649414063, "get_robot_state": 0.003628547191619873, "sim_render-ego0": 0.0037510514259338377, "get_duckie_state": 1.3065338134765626e-06, "in-drivable-lane": 2.6499999999999906, "deviation-heading": 0.9180096983869492, "agent_compute-ego0": 0.012004072666168214, "complete-iteration": 0.23075875759124756, "set_robot_commands": 0.002179560661315918, "deviation-center-line": 0.14190879982225874, "driven_lanedir_consec": 0.2850733588842411, "sim_compute_sim_state": 0.010201995372772216, "sim_compute_performance-ego0": 0.0019377565383911133}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.9661169475312417, "get_ui_image": 0.030548199764272786, "step_physics": 0.13701926805696435, "survival_time": 8.999999999999993, "driven_lanedir": 1.5523677567853218, "get_state_dump": 0.004664481674110033, "get_robot_state": 0.003834957575929758, "sim_render-ego0": 0.003973938483559624, "get_duckie_state": 1.4502699203912723e-06, "in-drivable-lane": 2.6500000000000075, "deviation-heading": 1.5605138970670358, "agent_compute-ego0": 0.012845634755508676, "complete-iteration": 0.20705016014984295, "set_robot_commands": 0.0021832766453864167, "deviation-center-line": 0.6108643967621402, "driven_lanedir_consec": 1.5523677567853218, "sim_compute_sim_state": 0.009741029686690694, "sim_compute_performance-ego0": 0.002139158670414877}, "LF-norm-small_loop-000-ego0": {"driven_any": 11.752950382102384, "get_ui_image": 0.025834162765299647, "step_physics": 0.1070363672448634, "survival_time": 59.99999999999873, "driven_lanedir": 4.505535089011568, "get_state_dump": 0.00449475340799527, "get_robot_state": 0.003576532986440031, "sim_render-ego0": 0.0037140365841982262, "get_duckie_state": 1.2653455646905573e-06, "in-drivable-lane": 36.04999999999909, "deviation-heading": 11.96763258431313, "agent_compute-ego0": 0.012107589262709034, "complete-iteration": 0.16709936151496577, "set_robot_commands": 0.002110616650609152, "deviation-center-line": 1.7857220375701073, "driven_lanedir_consec": 3.650215997531058, "sim_compute_sim_state": 0.006250974836198615, "sim_compute_performance-ego0": 0.001890345080309764}}
set_robot_commands_max0.002212159104449652
set_robot_commands_mean0.002171403265440285
set_robot_commands_median0.0021814186533511675
set_robot_commands_min0.002110616650609152
sim_compute_performance-ego0_max0.002151022662979891
sim_compute_performance-ego0_mean0.0020295707380239116
sim_compute_performance-ego0_median0.002038457604402995
sim_compute_performance-ego0_min0.001890345080309764
sim_compute_sim_state_max0.010532061638297216
sim_compute_sim_state_mean0.009181515383489686
sim_compute_sim_state_median0.009971512529731457
sim_compute_sim_state_min0.006250974836198615
sim_render-ego0_max0.003973938483559624
sim_render-ego0_mean0.00383187751047824
sim_render-ego0_median0.0038197674870775536
sim_render-ego0_min0.0037140365841982262
simulation-passed1
step_physics_max0.1566799259185791
step_physics_mean0.13110719034774346
step_physics_median0.13035623411376568
step_physics_min0.1070363672448634
survival_time_max59.99999999999873
survival_time_mean28.94999999999962
survival_time_min4.94999999999999
No reset possible
51042LFv-simerrorno0:02:07
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140632839825488
- M:video_aido:cmdline(in:/;out:/) 140632839826256
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51028LFv-simerrorno0:02:31
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140120074399504
- M:video_aido:cmdline(in:/;out:/) 140120074251424
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40519LFv-simsuccessno0:09:44
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40516LFv-simsuccessno0:08:13
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
39808LFv-simsuccessno0:10:46
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
39807LFv-simsuccessno0:09:15
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38576LFv-simsuccessno0:06:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38575LFv-simsuccessno0:07:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible