Duckietown Challenges Home Challenges Submissions

Submission 10757

Submission10757
Competingyes
Challengeaido5-LF-sim-validation
UserFernanda Custodio Pereira do Carmo 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 57910
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

57910

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
57910LFv-simsuccessyes0:16:01
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.9456208902982728
survival_time_median16.02500000000009
deviation-center-line_median0.7283008412920473
in-drivable-lane_median4.074999999999976


other stats
agent_compute-ego0_max0.014229064001394887
agent_compute-ego0_mean0.01346674039567466
agent_compute-ego0_median0.013586643856292366
agent_compute-ego0_min0.01246460986871902
complete-iteration_max0.21557253558619496
complete-iteration_mean0.1852329888982873
complete-iteration_median0.18354370902164205
complete-iteration_min0.1582720019636702
deviation-center-line_max4.4427925655218
deviation-center-line_mean1.6222908111648375
deviation-center-line_min0.5897689965534558
deviation-heading_max9.319730255856893
deviation-heading_mean3.344556994097948
deviation-heading_median1.4818142338036369
deviation-heading_min1.0948692529276258
driven_any_max10.228151299609324
driven_any_mean4.422815155491369
driven_any_median2.633675730981025
driven_any_min2.1957578603941013
driven_lanedir_consec_max9.281702538501603
driven_lanedir_consec_mean3.695300713949031
driven_lanedir_consec_min1.6082585366979725
driven_lanedir_max9.281702538501603
driven_lanedir_mean3.695300713949031
driven_lanedir_median1.9456208902982728
driven_lanedir_min1.6082585366979725
get_duckie_state_max1.6750210393084227e-06
get_duckie_state_mean1.5455411305855712e-06
get_duckie_state_median1.5850666914548896e-06
get_duckie_state_min1.3370101001240828e-06
get_robot_state_max0.004065891512397195
get_robot_state_mean0.0039223808207655255
get_robot_state_median0.003953731459195818
get_robot_state_min0.003716168852273273
get_state_dump_max0.005134002692034455
get_state_dump_mean0.004880434924443624
get_state_dump_median0.004873410548876198
get_state_dump_min0.004640915907987647
get_ui_image_max0.03630806150890532
get_ui_image_mean0.03114372187718835
get_ui_image_median0.030833670095591285
get_ui_image_min0.02659948580866551
in-drivable-lane_max8.100000000000115
in-drivable-lane_mean4.575000000000024
in-drivable-lane_min2.050000000000029
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 2.4161677039138394, "get_ui_image": 0.02835094928741455, "step_physics": 0.10336512892785732, "survival_time": 13.65000000000006, "driven_lanedir": 2.1325314299917437, "get_state_dump": 0.004815564538440565, "get_robot_state": 0.003893260538142963, "sim_render-ego0": 0.003997213649053644, "get_duckie_state": 1.6750210393084227e-06, "in-drivable-lane": 2.050000000000029, "deviation-heading": 1.3210413961826422, "agent_compute-ego0": 0.013442309233393982, "complete-iteration": 0.1725745227215064, "set_robot_commands": 0.00233352880408294, "deviation-center-line": 0.5897689965534558, "driven_lanedir_consec": 2.1325314299917437, "sim_compute_sim_state": 0.010088807474957766, "sim_compute_performance-ego0": 0.0021805084534805186}, "LF-norm-zigzag-000-ego0": {"driven_any": 2.1957578603941013, "get_ui_image": 0.03630806150890532, "step_physics": 0.13619318949122008, "survival_time": 14.650000000000071, "driven_lanedir": 1.758710350604802, "get_state_dump": 0.005134002692034455, "get_robot_state": 0.004065891512397195, "sim_render-ego0": 0.004098413752860764, "get_duckie_state": 1.5886462464624522e-06, "in-drivable-lane": 3.3500000000000476, "deviation-heading": 1.6425870714246311, "agent_compute-ego0": 0.01373097847919075, "complete-iteration": 0.21557253558619496, "set_robot_commands": 0.0023294914336431595, "deviation-center-line": 0.8334400842922647, "driven_lanedir_consec": 1.758710350604802, "sim_compute_sim_state": 0.011422321099002344, "sim_compute_performance-ego0": 0.002184588892930219}, "LF-norm-techtrack-000-ego0": {"driven_any": 2.8511837580482102, "get_ui_image": 0.03331639090376802, "step_physics": 0.11750307465692646, "survival_time": 17.400000000000112, "driven_lanedir": 1.6082585366979725, "get_state_dump": 0.004931256559311831, "get_robot_state": 0.004014202380248673, "sim_render-ego0": 0.004202107645379097, "get_duckie_state": 1.5814871364473271e-06, "in-drivable-lane": 8.100000000000115, "deviation-heading": 1.0948692529276258, "agent_compute-ego0": 0.014229064001394887, "complete-iteration": 0.1945128953217777, "set_robot_commands": 0.0024005266861792623, "deviation-center-line": 0.6231615982918298, "driven_lanedir_consec": 1.6082585366979725, "sim_compute_sim_state": 0.0115603908768356, "sim_compute_performance-ego0": 0.002245877738668447}, "LF-norm-small_loop-000-ego0": {"driven_any": 10.228151299609324, "get_ui_image": 0.02659948580866551, "step_physics": 0.09649491389526316, "survival_time": 59.99999999999873, "driven_lanedir": 9.281702538501603, "get_state_dump": 0.004640915907987647, "get_robot_state": 0.003716168852273273, "sim_render-ego0": 0.003814047718921569, "get_duckie_state": 1.3370101001240828e-06, "in-drivable-lane": 4.799999999999905, "deviation-heading": 9.319730255856893, "agent_compute-ego0": 0.01246460986871902, "complete-iteration": 0.1582720019636702, "set_robot_commands": 0.0022162336989505205, "deviation-center-line": 4.4427925655218, "driven_lanedir_consec": 9.281702538501603, "sim_compute_sim_state": 0.006243104640887639, "sim_compute_performance-ego0": 0.001990397705027305}}
set_robot_commands_max0.0024005266861792623
set_robot_commands_mean0.0023199451557139707
set_robot_commands_median0.00233151011886305
set_robot_commands_min0.0022162336989505205
sim_compute_performance-ego0_max0.002245877738668447
sim_compute_performance-ego0_mean0.0021503431975266225
sim_compute_performance-ego0_median0.0021825486732053685
sim_compute_performance-ego0_min0.001990397705027305
sim_compute_sim_state_max0.0115603908768356
sim_compute_sim_state_mean0.00982865602292084
sim_compute_sim_state_median0.010755564286980056
sim_compute_sim_state_min0.006243104640887639
sim_render-ego0_max0.004202107645379097
sim_render-ego0_mean0.004027945691553769
sim_render-ego0_median0.004047813700957204
sim_render-ego0_min0.003814047718921569
simulation-passed1
step_physics_max0.13619318949122008
step_physics_mean0.11338907674281676
step_physics_median0.1104341017923919
step_physics_min0.09649491389526316
survival_time_max59.99999999999873
survival_time_mean26.42499999999974
survival_time_min13.65000000000006
No reset possible
51773LFv-simerrorno0:07:54
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139753362526464
- M:video_aido:cmdline(in:/;out:/) 139753362897648
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51767LFv-simhost-errorno0:06:35
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51756LFv-simerrorno0:04:28
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139868690281136
- M:video_aido:cmdline(in:/;out:/) 139868690282672
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51746LFv-simerrorno0:08:44
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140453907317664
- M:video_aido:cmdline(in:/;out:/) 140453907232608
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
41005LFv-simsuccessno0:08:55
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
37112LFv-simsuccessno0:10:23
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
37111LFv-simsuccessno0:10:56
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible