Duckietown Challenges Home Challenges Submissions

Submission 11689

Submission11689
Competingyes
Challengeaido5-LF-sim-validation
UserMo Kleit 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 54024
Next
User labelreal-exercise-2
Admin priority50
Blessingn/a
User priority50

54024

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
54024LFv-simsuccessyes0:16:42
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median4.888701981206721
survival_time_median26.700000000000088
deviation-center-line_median1.19251596391243
in-drivable-lane_median10.550000000000022


other stats
agent_compute-ego0_max0.013022783289088648
agent_compute-ego0_mean0.012912543603924707
agent_compute-ego0_median0.012920137032327531
agent_compute-ego0_min0.01278711706195512
complete-iteration_max0.2829570422569911
complete-iteration_mean0.24750133363331123
complete-iteration_median0.25298740362010186
complete-iteration_min0.20107348503605013
deviation-center-line_max2.262849534868761
deviation-center-line_mean1.1905099224047069
deviation-center-line_min0.1141582269252077
deviation-heading_max9.943268617547533
deviation-heading_mean5.871773548591414
deviation-heading_median6.368633736465682
deviation-heading_min0.8065581038867561
driven_any_max15.159544096153176
driven_any_mean9.236123919686785
driven_any_median10.162275974117708
driven_any_min1.4603996343585504
driven_lanedir_consec_max8.998609010568613
driven_lanedir_consec_mean4.829326210978145
driven_lanedir_consec_min0.5412918709305261
driven_lanedir_max8.998609010568613
driven_lanedir_mean4.859431869923382
driven_lanedir_median4.946651592923067
driven_lanedir_min0.5458152832787806
get_duckie_state_max1.564621925354004e-06
get_duckie_state_mean1.3873942517963963e-06
get_duckie_state_median1.3967806805212712e-06
get_duckie_state_min1.1913937207890396e-06
get_robot_state_max0.003907748864674928
get_robot_state_mean0.003794297503357814
get_robot_state_median0.0037912953885789152
get_robot_state_min0.0036868503715984983
get_state_dump_max0.006271353215416156
get_state_dump_mean0.005189362070103244
get_state_dump_median0.0048561203810903765
get_state_dump_min0.004773854302816065
get_ui_image_max0.036399103701114655
get_ui_image_mean0.03183726503315551
get_ui_image_median0.03193124411274944
get_ui_image_min0.02708746820600851
in-drivable-lane_max15.849999999999886
in-drivable-lane_mean9.94999999999998
in-drivable-lane_min2.8499999999999908
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 14.032948782765946, "get_ui_image": 0.030121852013963404, "step_physics": 0.16589110803538715, "survival_time": 36.40000000000007, "driven_lanedir": 8.998609010568613, "get_state_dump": 0.004912005530463325, "get_robot_state": 0.003907748864674928, "sim_render-ego0": 0.003966908396026234, "get_duckie_state": 1.457329476648234e-06, "in-drivable-lane": 11.199999999999958, "deviation-heading": 9.3240090558775, "agent_compute-ego0": 0.01278711706195512, "complete-iteration": 0.2363897148310239, "set_robot_commands": 0.002448591185204777, "deviation-center-line": 2.262849534868761, "driven_lanedir_consec": 8.998609010568613, "sim_compute_sim_state": 0.010053606844406232, "sim_compute_performance-ego0": 0.0022066790707644447}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.4603996343585504, "get_ui_image": 0.036399103701114655, "step_physics": 0.20642392337322235, "survival_time": 4.749999999999991, "driven_lanedir": 0.5458152832787806, "get_state_dump": 0.004800235231717427, "get_robot_state": 0.00371419886747996, "sim_render-ego0": 0.003899720807870229, "get_duckie_state": 1.564621925354004e-06, "in-drivable-lane": 2.8499999999999908, "deviation-heading": 0.8065581038867561, "agent_compute-ego0": 0.013009930650393168, "complete-iteration": 0.2829570422569911, "set_robot_commands": 0.002345395584901174, "deviation-center-line": 0.1141582269252077, "driven_lanedir_consec": 0.5412918709305261, "sim_compute_sim_state": 0.01020933190981547, "sim_compute_performance-ego0": 0.0020623629291852317}, "LF-norm-techtrack-000-ego0": {"driven_any": 15.159544096153176, "get_ui_image": 0.033740636211535474, "step_physics": 0.1928279868247995, "survival_time": 39.3999999999999, "driven_lanedir": 7.741301393242652, "get_state_dump": 0.004773854302816065, "get_robot_state": 0.0038683919096778705, "sim_render-ego0": 0.003948114972906269, "get_duckie_state": 1.3362318843943085e-06, "in-drivable-lane": 15.849999999999886, "deviation-heading": 9.943268617547533, "agent_compute-ego0": 0.013022783289088648, "complete-iteration": 0.2695850924091798, "set_robot_commands": 0.002356534675046971, "deviation-center-line": 1.9066464938221457, "driven_lanedir_consec": 7.62540216980996, "sim_compute_sim_state": 0.012812912086419128, "sim_compute_performance-ego0": 0.002140206529160416}, "LF-norm-small_loop-000-ego0": {"driven_any": 6.291603165469468, "get_ui_image": 0.02708746820600851, "step_physics": 0.136509935177666, "survival_time": 17.000000000000107, "driven_lanedir": 2.152001792603483, "get_state_dump": 0.006271353215416156, "get_robot_state": 0.0036868503715984983, "sim_render-ego0": 0.003871659729138148, "get_duckie_state": 1.1913937207890396e-06, "in-drivable-lane": 9.900000000000084, "deviation-heading": 3.413258417053865, "agent_compute-ego0": 0.012830343414261891, "complete-iteration": 0.20107348503605013, "set_robot_commands": 0.0023991872837816164, "deviation-center-line": 0.4783854340027145, "driven_lanedir_consec": 2.152001792603483, "sim_compute_sim_state": 0.006275272788889597, "sim_compute_performance-ego0": 0.0020531608212378716}}
set_robot_commands_max0.002448591185204777
set_robot_commands_mean0.002387427182233635
set_robot_commands_median0.002377860979414294
set_robot_commands_min0.002345395584901174
sim_compute_performance-ego0_max0.0022066790707644447
sim_compute_performance-ego0_mean0.0021156023375869908
sim_compute_performance-ego0_median0.002101284729172824
sim_compute_performance-ego0_min0.0020531608212378716
sim_compute_sim_state_max0.012812912086419128
sim_compute_sim_state_mean0.009837780907382607
sim_compute_sim_state_median0.01013146937711085
sim_compute_sim_state_min0.006275272788889597
sim_render-ego0_max0.003966908396026234
sim_render-ego0_mean0.00392160097648522
sim_render-ego0_median0.003923917890388249
sim_render-ego0_min0.003871659729138148
simulation-passed1
step_physics_max0.20642392337322235
step_physics_mean0.17541323835276876
step_physics_median0.17935954743009336
step_physics_min0.136509935177666
survival_time_max39.3999999999999
survival_time_mean24.387500000000017
survival_time_min4.749999999999991
No reset possible
54019LFv-simsuccessyes0:08:55
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
54016LFv-simsuccessyes0:09:41
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
48980LFv-simerrorno0:03:11
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140560989992224
- M:video_aido:cmdline(in:/;out:/) 140560988933520
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43537LFv-simsuccessno0:11:56
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43534LFv-simfailedno0:01:10
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43532LFv-simfailedno0:01:22
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible