Duckietown Challenges Home Challenges Submissions

Submission 11552

Submission11552
Competingyes
Challengeaido5-LF-sim-validation
UserHimanshu Arora 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 54407
Next
User labelsim-exercise-2
Admin priority50
Blessingn/a
User priority50

54407

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
54407LFv-simsuccessyes0:22:33
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median3.0614340646808813
survival_time_median36.87499999999939
deviation-center-line_median2.164689240240502
in-drivable-lane_median10.699999999999712


other stats
agent_compute-ego0_max0.01291346327166691
agent_compute-ego0_mean0.012387284907964338
agent_compute-ego0_median0.012321448826790144
agent_compute-ego0_min0.011992778706610151
complete-iteration_max0.21083196791681424
complete-iteration_mean0.19264160686253523
complete-iteration_median0.19548877088321193
complete-iteration_min0.16875691776690277
deviation-center-line_max4.367551585062542
deviation-center-line_mean2.197158898900628
deviation-center-line_min0.09170553005896714
deviation-heading_max15.591283674524092
deviation-heading_mean8.08827776386047
deviation-heading_median7.933272968221055
deviation-heading_min0.895281444475679
driven_any_max17.826805572625638
driven_any_mean8.272459165141338
driven_any_median6.7817179531959875
driven_any_min1.6995951815477424
driven_lanedir_consec_max14.280914991766174
driven_lanedir_consec_mean5.128777261762024
driven_lanedir_consec_min0.11132592592015844
driven_lanedir_max14.525746620743494
driven_lanedir_mean6.004560339162782
driven_lanedir_median4.6905844049937375
driven_lanedir_min0.11132592592015844
get_duckie_state_max1.9021656202233357e-06
get_duckie_state_mean1.8636141332518904e-06
get_duckie_state_median1.8622737420756743e-06
get_duckie_state_min1.8277434286328777e-06
get_robot_state_max0.0037592269372248993
get_robot_state_mean0.0036920580111095658
get_robot_state_median0.003688363618279351
get_robot_state_min0.0036322778706546622
get_state_dump_max0.0047620301661284075
get_state_dump_mean0.004652776098787451
get_state_dump_median0.0046281623154985525
get_state_dump_min0.004592749598024291
get_ui_image_max0.03515358153826787
get_ui_image_mean0.030576550954200497
get_ui_image_median0.03001219874551467
get_ui_image_min0.027128224787504776
in-drivable-lane_max12.35000000000006
in-drivable-lane_mean10.537499999999875
in-drivable-lane_min8.400000000000016
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 17.826805572625638, "get_ui_image": 0.02774276364157341, "step_physics": 0.1225362127766224, "survival_time": 59.99999999999873, "driven_lanedir": 14.525746620743494, "get_state_dump": 0.004592749598024291, "get_robot_state": 0.0036322778706546622, "sim_render-ego0": 0.00365176208807368, "get_duckie_state": 1.882931076418252e-06, "in-drivable-lane": 10.749999999999648, "deviation-heading": 14.968975714154242, "agent_compute-ego0": 0.012128965542973527, "complete-iteration": 0.18770671426803243, "set_robot_commands": 0.002114266976031733, "deviation-center-line": 4.17469418711152, "driven_lanedir_consec": 14.280914991766174, "sim_compute_sim_state": 0.00929995659090498, "sim_compute_performance-ego0": 0.001924836169075311}, "LF-norm-zigzag-000-ego0": {"driven_any": 11.67529486924557, "get_ui_image": 0.03515358153826787, "step_physics": 0.13635979842186768, "survival_time": 59.99999999999873, "driven_lanedir": 8.827996385009971, "get_state_dump": 0.0046050945586904105, "get_robot_state": 0.003644531712146921, "sim_render-ego0": 0.003697016157774405, "get_duckie_state": 1.8277434286328777e-06, "in-drivable-lane": 10.649999999999777, "deviation-heading": 15.591283674524092, "agent_compute-ego0": 0.011992778706610151, "complete-iteration": 0.21083196791681424, "set_robot_commands": 0.00210155277426892, "deviation-center-line": 4.367551585062542, "driven_lanedir_consec": 5.577434721736097, "sim_compute_sim_state": 0.011235194837520958, "sim_compute_performance-ego0": 0.0019596738680316248}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.6995951815477424, "get_ui_image": 0.03228163384945593, "step_physics": 0.13331385862047426, "survival_time": 10.650000000000016, "driven_lanedir": 0.5531724249775042, "get_state_dump": 0.0046512300723066955, "get_robot_state": 0.0037321955244117807, "sim_render-ego0": 0.0038035528682102662, "get_duckie_state": 1.8416164077330973e-06, "in-drivable-lane": 8.400000000000016, "deviation-heading": 0.895281444475679, "agent_compute-ego0": 0.01291346327166691, "complete-iteration": 0.2032708274983914, "set_robot_commands": 0.0022204111669665186, "deviation-center-line": 0.15468429336948347, "driven_lanedir_consec": 0.5454334076256656, "sim_compute_sim_state": 0.008284864024581196, "sim_compute_performance-ego0": 0.0019851410500357084}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.8881410371464045, "get_ui_image": 0.027128224787504776, "step_physics": 0.10710737411526666, "survival_time": 13.75000000000006, "driven_lanedir": 0.11132592592015844, "get_state_dump": 0.0047620301661284075, "get_robot_state": 0.0037592269372248993, "sim_render-ego0": 0.003853099933568982, "get_duckie_state": 1.9021656202233357e-06, "in-drivable-lane": 12.35000000000006, "deviation-heading": 0.8975702222878674, "agent_compute-ego0": 0.01251393211060676, "complete-iteration": 0.16875691776690277, "set_robot_commands": 0.002281616563382356, "deviation-center-line": 0.09170553005896714, "driven_lanedir_consec": 0.11132592592015844, "sim_compute_sim_state": 0.005269232867420584, "sim_compute_performance-ego0": 0.0019951877386673636}}
set_robot_commands_max0.002281616563382356
set_robot_commands_mean0.002179461870162382
set_robot_commands_median0.002167339071499126
set_robot_commands_min0.00210155277426892
sim_compute_performance-ego0_max0.0019951877386673636
sim_compute_performance-ego0_mean0.001966209706452502
sim_compute_performance-ego0_median0.001972407459033667
sim_compute_performance-ego0_min0.001924836169075311
sim_compute_sim_state_max0.011235194837520958
sim_compute_sim_state_mean0.00852231208010693
sim_compute_sim_state_median0.008792410307743088
sim_compute_sim_state_min0.005269232867420584
sim_render-ego0_max0.003853099933568982
sim_render-ego0_mean0.003751357761906833
sim_render-ego0_median0.003750284512992336
sim_render-ego0_min0.00365176208807368
simulation-passed1
step_physics_max0.13635979842186768
step_physics_mean0.12482931098355776
step_physics_median0.12792503569854835
step_physics_min0.10710737411526666
survival_time_max59.99999999999873
survival_time_mean36.09999999999938
survival_time_min10.650000000000016
No reset possible
54403LFv-simsuccessyes0:26:14
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49574LFv-simerrorno0:07:08
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140030418091216
- M:video_aido:cmdline(in:/;out:/) 140030418090016
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49570LFv-simerrorno0:07:47
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140021038540352
- M:video_aido:cmdline(in:/;out:/) 140021038458288
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49563LFv-simerrorno0:09:21
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140583722483472
- M:video_aido:cmdline(in:/;out:/) 140583722494464
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42867LFv-simsuccessno0:10:52
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible