Duckietown Challenges Home Challenges Submissions

Submission 10747

Submission10747
Competingyes
Challengeaido5-LF-sim-validation
UserPhilippe Reddy 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 57924
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

57924

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
57924LFv-simsuccessyes0:27:12
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.6958840493919114
survival_time_median59.99999999999873
deviation-center-line_median2.929105587604598
in-drivable-lane_median4.274999999999757


other stats
agent_compute-ego0_max0.012667972636672686
agent_compute-ego0_mean0.01217018144963316
agent_compute-ego0_median0.012193490424620718
agent_compute-ego0_min0.011625772312618512
complete-iteration_max0.20347232227023693
complete-iteration_mean0.1755758875727648
complete-iteration_median0.18018325915941977
complete-iteration_min0.13846470970198277
deviation-center-line_max5.173006060111576
deviation-center-line_mean2.876204819578066
deviation-center-line_min0.4736020429914912
deviation-heading_max13.363339361617845
deviation-heading_mean7.104469958665213
deviation-heading_median6.0564647948506
deviation-heading_min2.9416108833418093
driven_any_max3.4412105262142565
driven_any_mean1.9602974517161993
driven_any_median1.8432469207926117
driven_any_min0.7134854390653182
driven_lanedir_consec_max2.976305685311349
driven_lanedir_consec_mean1.6597670697090594
driven_lanedir_consec_min0.2709944947410654
driven_lanedir_max2.976305685311349
driven_lanedir_mean1.6597670697090594
driven_lanedir_median1.6958840493919114
driven_lanedir_min0.2709944947410654
get_duckie_state_max1.1416696489700965e-06
get_duckie_state_mean1.1212999608371252e-06
get_duckie_state_median1.1312040722986486e-06
get_duckie_state_min1.0811220497811068e-06
get_robot_state_max0.003631266427973129
get_robot_state_mean0.0035424849042127904
get_robot_state_median0.0035168838076225697
get_robot_state_min0.003504905573632894
get_state_dump_max0.004471749092121108
get_state_dump_mean0.004382041844074671
get_state_dump_median0.004355325190649739
get_state_dump_min0.004345767902878096
get_ui_image_max0.03575790603205843
get_ui_image_mean0.030386696473772283
get_ui_image_median0.03013621063454761
get_ui_image_min0.025516458593935493
in-drivable-lane_max6.650000000000068
in-drivable-lane_mean3.799999999999895
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 0.9499546039787348, "get_ui_image": 0.028100203514893188, "step_physics": 0.13870313244993543, "survival_time": 59.99999999999873, "driven_lanedir": 0.9428498039387444, "get_state_dump": 0.004471749092121108, "get_robot_state": 0.003631266427973129, "sim_render-ego0": 0.0037294387023315938, "get_duckie_state": 1.1416696489700965e-06, "in-drivable-lane": 0.0, "deviation-heading": 4.48423992851548, "agent_compute-ego0": 0.011625772312618512, "complete-iteration": 0.20347232227023693, "set_robot_commands": 0.0021926018717286985, "deviation-center-line": 2.7406482542628523, "driven_lanedir_consec": 0.9428498039387444, "sim_compute_sim_state": 0.009019159059738937, "sim_compute_performance-ego0": 0.0019218133550003905}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.7134854390653182, "get_ui_image": 0.03575790603205843, "step_physics": 0.10377436763835404, "survival_time": 13.200000000000053, "driven_lanedir": 0.2709944947410654, "get_state_dump": 0.004345767902878096, "get_robot_state": 0.003527832930942751, "sim_render-ego0": 0.003648428647023327, "get_duckie_state": 1.123716246406987e-06, "in-drivable-lane": 6.650000000000068, "deviation-heading": 2.9416108833418093, "agent_compute-ego0": 0.012667972636672686, "complete-iteration": 0.1785239003739267, "set_robot_commands": 0.0020797990403085384, "deviation-center-line": 0.4736020429914912, "driven_lanedir_consec": 0.2709944947410654, "sim_compute_sim_state": 0.010775875595380674, "sim_compute_performance-ego0": 0.0018706033814628168}, "LF-norm-techtrack-000-ego0": {"driven_any": 2.7365392376064888, "get_ui_image": 0.03217221775420202, "step_physics": 0.11085173271776336, "survival_time": 59.99999999999873, "driven_lanedir": 2.448918294845078, "get_state_dump": 0.004355915380854293, "get_robot_state": 0.003504905573632894, "sim_render-ego0": 0.003649488277578235, "get_duckie_state": 1.0811220497811068e-06, "in-drivable-lane": 3.3499999999998096, "deviation-heading": 7.628689661185718, "agent_compute-ego0": 0.012608919413659495, "complete-iteration": 0.18184261794491272, "set_robot_commands": 0.0022234894850172667, "deviation-center-line": 3.117562920946344, "driven_lanedir_consec": 2.448918294845078, "sim_compute_sim_state": 0.010547907723673774, "sim_compute_performance-ego0": 0.0018517957142648051}, "LF-norm-small_loop-000-ego0": {"driven_any": 3.4412105262142565, "get_ui_image": 0.025516458593935493, "step_physics": 0.0797393101637409, "survival_time": 59.99999999999873, "driven_lanedir": 2.976305685311349, "get_state_dump": 0.004354735000445186, "get_robot_state": 0.003505934684302388, "sim_render-ego0": 0.003597113611696166, "get_duckie_state": 1.13869189819031e-06, "in-drivable-lane": 5.199999999999704, "deviation-heading": 13.363339361617845, "agent_compute-ego0": 0.01177806143558194, "complete-iteration": 0.13846470970198277, "set_robot_commands": 0.002072077607433564, "deviation-center-line": 5.173006060111576, "driven_lanedir_consec": 2.976305685311349, "sim_compute_sim_state": 0.00597327079105933, "sim_compute_performance-ego0": 0.001851162445932304}}
set_robot_commands_max0.0022234894850172667
set_robot_commands_mean0.0021419920011220167
set_robot_commands_median0.0021362004560186183
set_robot_commands_min0.002072077607433564
sim_compute_performance-ego0_max0.0019218133550003905
sim_compute_performance-ego0_mean0.001873843724165079
sim_compute_performance-ego0_median0.0018611995478638112
sim_compute_performance-ego0_min0.001851162445932304
sim_compute_sim_state_max0.010775875595380674
sim_compute_sim_state_mean0.00907905329246318
sim_compute_sim_state_median0.009783533391706355
sim_compute_sim_state_min0.00597327079105933
sim_render-ego0_max0.0037294387023315938
sim_render-ego0_mean0.0036561173096573306
sim_render-ego0_median0.003648958462300781
sim_render-ego0_min0.003597113611696166
simulation-passed1
step_physics_max0.13870313244993543
step_physics_mean0.10826713574244844
step_physics_median0.1073130501780587
step_physics_min0.0797393101637409
survival_time_max59.99999999999873
survival_time_mean48.29999999999906
survival_time_min13.200000000000053
No reset possible
51796LFv-simerrorno0:08:48
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140440217823072
- M:video_aido:cmdline(in:/;out:/) 140440217821872
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51788LFv-simerrorno0:09:01
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139818211133040
- M:video_aido:cmdline(in:/;out:/) 139818211562208
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51785LFv-simerrorno0:09:37
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140164079227856
- M:video_aido:cmdline(in:/;out:/) 140164079139136
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
41012LFv-simsuccessno0:08:46
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
41011LFv-simsuccessno0:08:42
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
36770LFv-simsuccessno0:10:51
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
36769LFv-simsuccessno0:11:03
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible