Duckietown Challenges Home Challenges Submissions

Submission 10901

Submission10901
Competingyes
Challengeaido5-LF-sim-validation
UserAyman Shams 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 57529
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

57529

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
57529LFv-simsuccessyes0:05:52
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.2454069312858963
survival_time_median8.24999999999999
deviation-center-line_median0.12687610248590775
in-drivable-lane_median6.52499999999999


other stats
agent_compute-ego0_max0.012952254000437789
agent_compute-ego0_mean0.012631866728846511
agent_compute-ego0_median0.01259678412790168
agent_compute-ego0_min0.012381644659144904
complete-iteration_max0.2731310367584229
complete-iteration_mean0.2042988671491283
complete-iteration_median0.189403240286675
complete-iteration_min0.16525795126474033
deviation-center-line_max0.35740040092068803
deviation-center-line_mean0.17184268989688783
deviation-center-line_min0.07621815369504775
deviation-heading_max1.6102083672622918
deviation-heading_mean0.9388928790031744
deviation-heading_median0.7606538511391056
deviation-heading_min0.6240554464721946
driven_any_max2.3815392205161308
driven_any_mean1.4224076859833308
driven_any_median1.506245595395628
driven_any_min0.2956003326259363
driven_lanedir_consec_max0.5290458888382591
driven_lanedir_consec_mean0.28971810080383675
driven_lanedir_consec_min0.1390126518052952
driven_lanedir_max0.5290458888382591
driven_lanedir_mean0.28971810080383675
driven_lanedir_median0.2454069312858963
driven_lanedir_min0.1390126518052952
get_duckie_state_max1.6641425320422314e-06
get_duckie_state_mean1.5857442583421382e-06
get_duckie_state_median1.579253904281124e-06
get_duckie_state_min1.5203266927640732e-06
get_robot_state_max0.003957676887512207
get_robot_state_mean0.003876255277038435
get_robot_state_median0.003914582730651249
get_robot_state_min0.003718178759339035
get_state_dump_max0.005261330604553222
get_state_dump_mean0.00500898792874649
get_state_dump_median0.005017013287343568
get_state_dump_min0.0047405945357456005
get_ui_image_max0.03713870048522949
get_ui_image_mean0.031113954962416812
get_ui_image_median0.030770265928830764
get_ui_image_min0.025776587506776215
in-drivable-lane_max9.100000000000016
in-drivable-lane_mean5.837499999999999
in-drivable-lane_min1.1999999999999988
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 2.3815392205161308, "get_ui_image": 0.028823919564365863, "step_physics": 0.1030913471696846, "survival_time": 12.40000000000004, "driven_lanedir": 0.5290458888382591, "get_state_dump": 0.00510681106383542, "get_robot_state": 0.003934299131952615, "sim_render-ego0": 0.004167038752850759, "get_duckie_state": 1.6641425320422314e-06, "in-drivable-lane": 9.100000000000016, "deviation-heading": 1.6102083672622918, "agent_compute-ego0": 0.012952254000437789, "complete-iteration": 0.1728571332602137, "set_robot_commands": 0.002407303775649473, "deviation-center-line": 0.35740040092068803, "driven_lanedir_consec": 0.5290458888382591, "sim_compute_sim_state": 0.010088365240748147, "sim_compute_performance-ego0": 0.002186173894797942}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.2956003326259363, "get_ui_image": 0.03713870048522949, "step_physics": 0.19502700805664064, "survival_time": 2.4499999999999993, "driven_lanedir": 0.1390126518052952, "get_state_dump": 0.005261330604553222, "get_robot_state": 0.003957676887512207, "sim_render-ego0": 0.0042959976196289065, "get_duckie_state": 1.6164779663085938e-06, "in-drivable-lane": 1.1999999999999988, "deviation-heading": 0.8288989784941195, "agent_compute-ego0": 0.012720723152160643, "complete-iteration": 0.2731310367584229, "set_robot_commands": 0.002362475395202637, "deviation-center-line": 0.07621815369504775, "driven_lanedir_consec": 0.1390126518052952, "sim_compute_sim_state": 0.009965167045593262, "sim_compute_performance-ego0": 0.0023007774353027343}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.2913632033705504, "get_ui_image": 0.032716612293295665, "step_physics": 0.13522482898137342, "survival_time": 7.249999999999982, "driven_lanedir": 0.19551066567029096, "get_state_dump": 0.004927215510851716, "get_robot_state": 0.003894866329349883, "sim_render-ego0": 0.003967249230162738, "get_duckie_state": 1.5203266927640732e-06, "in-drivable-lane": 5.6999999999999815, "deviation-heading": 0.6924087237840916, "agent_compute-ego0": 0.012472845103642712, "complete-iteration": 0.20594934731313627, "set_robot_commands": 0.0023511060296672664, "deviation-center-line": 0.08342321756054183, "driven_lanedir_consec": 0.19551066567029096, "sim_compute_sim_state": 0.008255821384795725, "sim_compute_performance-ego0": 0.0020467418513885915}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.7211279874207055, "get_ui_image": 0.025776587506776215, "step_physics": 0.10496590086208876, "survival_time": 9.249999999999996, "driven_lanedir": 0.29530319690150164, "get_state_dump": 0.0047405945357456005, "get_robot_state": 0.003718178759339035, "sim_render-ego0": 0.0038867522311467, "get_duckie_state": 1.5420298422536542e-06, "in-drivable-lane": 7.349999999999998, "deviation-heading": 0.6240554464721946, "agent_compute-ego0": 0.012381644659144904, "complete-iteration": 0.16525795126474033, "set_robot_commands": 0.002299893286920363, "deviation-center-line": 0.17032898741127367, "driven_lanedir_consec": 0.29530319690150164, "sim_compute_sim_state": 0.005375049447500577, "sim_compute_performance-ego0": 0.002016449487337502}}
set_robot_commands_max0.002407303775649473
set_robot_commands_mean0.0023551946218599348
set_robot_commands_median0.0023567907124349514
set_robot_commands_min0.002299893286920363
sim_compute_performance-ego0_max0.0023007774353027343
sim_compute_performance-ego0_mean0.002137535667206692
sim_compute_performance-ego0_median0.0021164578730932665
sim_compute_performance-ego0_min0.002016449487337502
sim_compute_sim_state_max0.010088365240748147
sim_compute_sim_state_mean0.008421100779659428
sim_compute_sim_state_median0.009110494215194494
sim_compute_sim_state_min0.005375049447500577
sim_render-ego0_max0.0042959976196289065
sim_render-ego0_mean0.004079259458447276
sim_render-ego0_median0.004067143991506749
sim_render-ego0_min0.0038867522311467
simulation-passed1
step_physics_max0.19502700805664064
step_physics_mean0.13457727126744687
step_physics_median0.12009536492173108
step_physics_min0.1030913471696846
survival_time_max12.40000000000004
survival_time_mean7.837500000000005
survival_time_min2.4499999999999993
No reset possible
51273LFv-simerrorno0:06:13
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140706971757248
- M:video_aido:cmdline(in:/;out:/) 140706971754752
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40634LFv-simsuccessno0:09:24
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38385LFv-simsuccessno0:09:21
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible