Duckietown Challenges Home Challenges Submissions

Submission 10815

Submission10815
Competingyes
Challengeaido5-LF-sim-validation
UserYishu Malhotra 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 57811
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

57811

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
57811LFv-simsuccessyes0:11:50
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.29090468748202014
survival_time_median14.775000000000077
deviation-center-line_median0.1956832410145883
in-drivable-lane_median10.250000000000078


other stats
agent_compute-ego0_max0.01504557354505672
agent_compute-ego0_mean0.013566519560831722
agent_compute-ego0_median0.01369328788321351
agent_compute-ego0_min0.01183392893184315
complete-iteration_max0.2297050343003384
complete-iteration_mean0.19524042896539215
complete-iteration_median0.20434828359418503
complete-iteration_min0.1425601143728603
deviation-center-line_max0.3932482026957615
deviation-center-line_mean0.24064326110719303
deviation-center-line_min0.17795835970383397
deviation-heading_max3.2275172004075325
deviation-heading_mean1.5182173981973723
deviation-heading_median1.091263876436282
deviation-heading_min0.6628246395093924
driven_any_max6.1238312212059265
driven_any_mean2.4099600464193935
driven_any_median1.5978271189242632
driven_any_min0.32035472662312225
driven_lanedir_consec_max0.6579578437833231
driven_lanedir_consec_mean0.3423820819161144
driven_lanedir_consec_min0.12976110891709425
driven_lanedir_max0.7892641032748131
driven_lanedir_mean0.3752086467889869
driven_lanedir_median0.29090468748202014
driven_lanedir_min0.12976110891709425
get_duckie_state_max2.8133392333984377e-06
get_duckie_state_mean2.5040703723041873e-06
get_duckie_state_median2.59632007283325e-06
get_duckie_state_min2.010302110151811e-06
get_robot_state_max0.004487415839885843
get_robot_state_mean0.00394067510203308
get_robot_state_median0.003888049264637765
get_robot_state_min0.003499186038970947
get_state_dump_max0.005550767087388313
get_state_dump_mean0.0049537501531188395
get_state_dump_median0.004905280731410043
get_state_dump_min0.004453672062266957
get_ui_image_max0.03600262937874629
get_ui_image_mean0.031491231407018934
get_ui_image_median0.032526561309914395
get_ui_image_min0.02490917362950065
in-drivable-lane_max41.09999999999964
in-drivable-lane_mean15.887499999999946
in-drivable-lane_min1.949999999999993
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 0.8408914486687298, "get_ui_image": 0.029598533352719076, "step_physics": 0.11735869359366502, "survival_time": 7.84999999999998, "driven_lanedir": 0.12976110891709425, "get_state_dump": 0.0049581029747105855, "get_robot_state": 0.003886840011500105, "sim_render-ego0": 0.004283698299263097, "get_duckie_state": 2.6920173741594145e-06, "in-drivable-lane": 6.24999999999998, "deviation-heading": 0.7721151888523583, "agent_compute-ego0": 0.013515144963807698, "complete-iteration": 0.18764066545269156, "set_robot_commands": 0.0024008102054837383, "deviation-center-line": 0.17795835970383397, "driven_lanedir_consec": 0.12976110891709425, "sim_compute_sim_state": 0.009256338771385484, "sim_compute_performance-ego0": 0.0022797116750403294}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.32035472662312225, "get_ui_image": 0.03545458926710972, "step_physics": 0.15003207949704905, "survival_time": 4.249999999999993, "driven_lanedir": 0.17078380781674118, "get_state_dump": 0.0048524584881095, "get_robot_state": 0.003889258517775425, "sim_render-ego0": 0.004287400911020678, "get_duckie_state": 2.500622771507086e-06, "in-drivable-lane": 1.949999999999993, "deviation-heading": 1.410412564020206, "agent_compute-ego0": 0.01504557354505672, "complete-iteration": 0.2297050343003384, "set_robot_commands": 0.002444683119308117, "deviation-center-line": 0.179723435057237, "driven_lanedir_consec": 0.17078380781674118, "sim_compute_sim_state": 0.011343304501023404, "sim_compute_performance-ego0": 0.0022514282270919444}, "LF-norm-techtrack-000-ego0": {"driven_any": 2.3547627891797966, "get_ui_image": 0.03600262937874629, "step_physics": 0.1414926304214302, "survival_time": 21.700000000000173, "driven_lanedir": 0.7892641032748131, "get_state_dump": 0.005550767087388313, "get_robot_state": 0.004487415839885843, "sim_render-ego0": 0.004660187096431337, "get_duckie_state": 2.8133392333984377e-06, "in-drivable-lane": 14.250000000000178, "deviation-heading": 3.2275172004075325, "agent_compute-ego0": 0.01387143080261932, "complete-iteration": 0.2210559017356785, "set_robot_commands": 0.002619059880574544, "deviation-center-line": 0.3932482026957615, "driven_lanedir_consec": 0.6579578437833231, "sim_compute_sim_state": 0.009798360693043675, "sim_compute_performance-ego0": 0.0024582681984737}, "LF-norm-small_loop-000-ego0": {"driven_any": 6.1238312212059265, "get_ui_image": 0.02490917362950065, "step_physics": 0.08437826037406922, "survival_time": 43.94999999999964, "driven_lanedir": 0.4110255671472991, "get_state_dump": 0.004453672062266957, "get_robot_state": 0.003499186038970947, "sim_render-ego0": 0.0036868669769980697, "get_duckie_state": 2.010302110151811e-06, "in-drivable-lane": 41.09999999999964, "deviation-heading": 0.6628246395093924, "agent_compute-ego0": 0.01183392893184315, "complete-iteration": 0.1425601143728603, "set_robot_commands": 0.0020598620176315308, "deviation-center-line": 0.21164304697193967, "driven_lanedir_consec": 0.4110255671472991, "sim_compute_sim_state": 0.005819011818278919, "sim_compute_performance-ego0": 0.0018377092751589688}}
set_robot_commands_max0.002619059880574544
set_robot_commands_mean0.002381103805749482
set_robot_commands_median0.0024227466623959275
set_robot_commands_min0.0020598620176315308
sim_compute_performance-ego0_max0.0024582681984737
sim_compute_performance-ego0_mean0.0022067793439412356
sim_compute_performance-ego0_median0.0022655699510661367
sim_compute_performance-ego0_min0.0018377092751589688
sim_compute_sim_state_max0.011343304501023404
sim_compute_sim_state_mean0.009054253945932868
sim_compute_sim_state_median0.009527349732214578
sim_compute_sim_state_min0.005819011818278919
sim_render-ego0_max0.004660187096431337
sim_render-ego0_mean0.004229538320928296
sim_render-ego0_median0.004285549605141888
sim_render-ego0_min0.0036868669769980697
simulation-passed1
step_physics_max0.15003207949704905
step_physics_mean0.12331541597155336
step_physics_median0.12942566200754765
step_physics_min0.08437826037406922
survival_time_max43.94999999999964
survival_time_mean19.437499999999947
survival_time_min4.249999999999993
No reset possible
51621LFv-simerrorno0:05:11
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140084851986336
- M:video_aido:cmdline(in:/;out:/) 140084851984416
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51613LFv-simhost-errorno0:03:21
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51608LFv-simhost-errorno0:03:26
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51601LFv-simerrorno0:03:38
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140280607231088
- M:video_aido:cmdline(in:/;out:/) 140278777848064
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40924LFv-simsuccessno0:10:54
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40922LFv-simsuccessno0:09:18
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40909LFv-simsuccessno0:10:54
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40908LFv-simsuccessno0:11:03
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38022LFv-simsuccessno0:10:45
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible