Duckietown Challenges Home Challenges Submissions

Submission 10839

Submission10839
Competingyes
Challengeaido5-LF-sim-validation
UserYishu Malhotra 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 57767
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

57767

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
57767LFv-simsuccessyes0:29:01
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median2.90490775427406
survival_time_median55.72499999999897
deviation-center-line_median1.1775391795451378
in-drivable-lane_median22.049999999999983


other stats
agent_compute-ego0_max0.01365934190964988
agent_compute-ego0_mean0.012781030057865102
agent_compute-ego0_median0.012655205499341246
agent_compute-ego0_min0.01215436732312804
complete-iteration_max0.1929681803159532
complete-iteration_mean0.17598851229771725
complete-iteration_median0.1780987291701734
complete-iteration_min0.15478841053456888
deviation-center-line_max3.968727544554039
deviation-center-line_mean1.8316256522075425
deviation-center-line_min1.0026967051858546
deviation-heading_max13.958914977187966
deviation-heading_mean6.8482128326317
deviation-heading_median4.909261219693025
deviation-heading_min3.615413913952783
driven_any_max7.10060827473819
driven_any_mean5.9557388250748495
driven_any_median6.665064710508713
driven_any_min3.3922176045437813
driven_lanedir_consec_max6.792997252177034
driven_lanedir_consec_mean3.5390935880494983
driven_lanedir_consec_min1.553561591472841
driven_lanedir_max6.792997252177034
driven_lanedir_mean3.551452129767121
driven_lanedir_median2.929624837709305
driven_lanedir_min1.553561591472841
get_duckie_state_max1.3939843983773289e-06
get_duckie_state_mean1.3224667849851116e-06
get_duckie_state_median1.313018016618413e-06
get_duckie_state_min1.269846708326292e-06
get_robot_state_max0.003938289201032351
get_robot_state_mean0.003836990371917832
get_robot_state_median0.0038772038136928822
get_robot_state_min0.003655264659253214
get_state_dump_max0.004999456207301836
get_state_dump_mean0.00484652395315853
get_state_dump_median0.004854893915658263
get_state_dump_min0.00467685177401576
get_ui_image_max0.03536095397047909
get_ui_image_mean0.031475260889222506
get_ui_image_median0.03187879769183542
get_ui_image_min0.02678249420274009
in-drivable-lane_max41.1999999999989
in-drivable-lane_mean21.58749999999972
in-drivable-lane_min1.050000000000015
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 6.913263593227138, "get_ui_image": 0.029034756684369695, "step_physics": 0.09901752865967534, "survival_time": 53.79999999999908, "driven_lanedir": 3.988441419296376, "get_state_dump": 0.004891280454956168, "get_robot_state": 0.003842070906275162, "sim_render-ego0": 0.004059491604704932, "get_duckie_state": 1.3209319048273198e-06, "in-drivable-lane": 27.89999999999975, "deviation-heading": 3.615413913952783, "agent_compute-ego0": 0.012824806994382385, "complete-iteration": 0.16950133517592067, "set_robot_commands": 0.002300434015144766, "deviation-center-line": 1.0026967051858546, "driven_lanedir_consec": 3.988441419296376, "sim_compute_sim_state": 0.011333858955758749, "sim_compute_performance-ego0": 0.002111826545127366}, "LF-norm-zigzag-000-ego0": {"driven_any": 3.3922176045437813, "get_ui_image": 0.03536095397047909, "step_physics": 0.11194149031615296, "survival_time": 30.00000000000029, "driven_lanedir": 1.553561591472841, "get_state_dump": 0.00467685177401576, "get_robot_state": 0.003655264659253214, "sim_render-ego0": 0.003890574672654544, "get_duckie_state": 1.269846708326292e-06, "in-drivable-lane": 16.200000000000212, "deviation-heading": 4.312309821063672, "agent_compute-ego0": 0.01215436732312804, "complete-iteration": 0.18669612316442608, "set_robot_commands": 0.0021691790436349574, "deviation-center-line": 1.032722689469178, "driven_lanedir_consec": 1.553561591472841, "sim_compute_sim_state": 0.01079453882480818, "sim_compute_performance-ego0": 0.001970042008131792}, "LF-norm-techtrack-000-ego0": {"driven_any": 6.416865827790288, "get_ui_image": 0.03472283869930115, "step_physics": 0.11604777572473166, "survival_time": 57.64999999999886, "driven_lanedir": 1.8708082561222343, "get_state_dump": 0.004999456207301836, "get_robot_state": 0.003938289201032351, "sim_render-ego0": 0.004275737023642819, "get_duckie_state": 1.3051041284095065e-06, "in-drivable-lane": 41.1999999999989, "deviation-heading": 5.506212618322379, "agent_compute-ego0": 0.01365934190964988, "complete-iteration": 0.1929681803159532, "set_robot_commands": 0.002324509248783105, "deviation-center-line": 1.3223556696210974, "driven_lanedir_consec": 1.8213740892517432, "sim_compute_sim_state": 0.0106991029488152, "sim_compute_performance-ego0": 0.002212193437837439}, "LF-norm-small_loop-000-ego0": {"driven_any": 7.10060827473819, "get_ui_image": 0.02678249420274009, "step_physics": 0.09179470044786388, "survival_time": 59.99999999999873, "driven_lanedir": 6.792997252177034, "get_state_dump": 0.0048185073763603575, "get_robot_state": 0.003912336721110602, "sim_render-ego0": 0.003997065443281925, "get_duckie_state": 1.3939843983773289e-06, "in-drivable-lane": 1.050000000000015, "deviation-heading": 13.958914977187966, "agent_compute-ego0": 0.012485604004300108, "complete-iteration": 0.15478841053456888, "set_robot_commands": 0.0023301036828364263, "deviation-center-line": 3.968727544554039, "driven_lanedir_consec": 6.792997252177034, "sim_compute_sim_state": 0.006462333402863946, "sim_compute_performance-ego0": 0.002120969496003595}}
set_robot_commands_max0.0023301036828364263
set_robot_commands_mean0.002281056497599814
set_robot_commands_median0.0023124716319639356
set_robot_commands_min0.0021691790436349574
sim_compute_performance-ego0_max0.002212193437837439
sim_compute_performance-ego0_mean0.002103757871775048
sim_compute_performance-ego0_median0.00211639802056548
sim_compute_performance-ego0_min0.001970042008131792
sim_compute_sim_state_max0.011333858955758749
sim_compute_sim_state_mean0.009822458533061518
sim_compute_sim_state_median0.01074682088681169
sim_compute_sim_state_min0.006462333402863946
sim_render-ego0_max0.004275737023642819
sim_render-ego0_mean0.004055717186071056
sim_render-ego0_median0.004028278523993428
sim_render-ego0_min0.003890574672654544
simulation-passed1
step_physics_max0.11604777572473166
step_physics_mean0.10470037378710596
step_physics_median0.10547950948791417
step_physics_min0.09179470044786388
survival_time_max59.99999999999873
survival_time_mean50.36249999999924
survival_time_min30.00000000000029
No reset possible
51549LFv-simerrorno0:07:25
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140250746620320
- M:video_aido:cmdline(in:/;out:/) 140250747492048
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51545LFv-simerrorno0:09:27
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140458045043904
- M:video_aido:cmdline(in:/;out:/) 140457723827344
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51535LFv-simerrorno0:08:52
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140144987428752
- M:video_aido:cmdline(in:/;out:/) 140144987429520
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40866LFv-simsuccessno0:09:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38089LFv-simsuccessno0:11:43
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible