Duckietown Challenges Home Challenges Submissions

Submission 11762

Submission11762
Competingyes
Challengeaido5-LF-sim-validation
UserOlga (Ge Ya) Xu 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 53805
Next
User labelsim-exercise-2
Admin priority50
Blessingn/a
User priority50

53805

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
53805LFv-simsuccessyes0:28:31
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.1062670700868882
survival_time_median59.99999999999873
deviation-center-line_median0.5970838990276617
in-drivable-lane_median45.4249999999992


other stats
agent_compute-ego0_max0.013243740958136482
agent_compute-ego0_mean0.012604011489116504
agent_compute-ego0_median0.012552947426318723
agent_compute-ego0_min0.012066410145692088
complete-iteration_max0.22077808251252048
complete-iteration_mean0.19644444490591295
complete-iteration_median0.1933289308730609
complete-iteration_min0.17834183536500955
deviation-center-line_max1.5947794297543822
deviation-center-line_mean0.7345984956086828
deviation-center-line_min0.14944675462502582
deviation-heading_max3.885930775708089
deviation-heading_mean2.4036262691831203
deviation-heading_median2.5672098264743957
deviation-heading_min0.5941546480756017
driven_any_max12.583237401491376
driven_any_mean9.683728950541829
driven_any_median12.218822071832363
driven_any_min1.714034257011202
driven_lanedir_consec_max3.537847072747071
driven_lanedir_consec_mean1.5235411912992285
driven_lanedir_consec_min0.34378355227606616
driven_lanedir_max3.537847072747071
driven_lanedir_mean1.5235411912992285
driven_lanedir_median1.1062670700868882
driven_lanedir_min0.34378355227606616
get_duckie_state_max1.710170024150127e-06
get_duckie_state_mean1.5937289697612027e-06
get_duckie_state_median1.5837663814090474e-06
get_duckie_state_min1.49721309207659e-06
get_robot_state_max0.0043805006388071424
get_robot_state_mean0.004239985179195047
get_robot_state_median0.0042473238572589955
get_robot_state_min0.004084792363455055
get_state_dump_max0.005472883018287453
get_state_dump_mean0.005335049123852244
get_state_dump_median0.005326587691295157
get_state_dump_min0.005214138094531209
get_ui_image_max0.039584274549742005
get_ui_image_mean0.03307011377495683
get_ui_image_median0.032280803223037405
get_ui_image_min0.02813457410401051
in-drivable-lane_max57.79999999999873
in-drivable-lane_mean38.73749999999929
in-drivable-lane_min6.300000000000001
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 12.583237401491376, "get_ui_image": 0.029882544581836507, "step_physics": 0.11501538227440215, "survival_time": 59.99999999999873, "driven_lanedir": 3.537847072747071, "get_state_dump": 0.005214138094531209, "get_robot_state": 0.004084792363455055, "sim_render-ego0": 0.0040932853057124435, "get_duckie_state": 1.547239305177001e-06, "in-drivable-lane": 41.1499999999993, "deviation-heading": 3.885930775708089, "agent_compute-ego0": 0.012066410145692088, "complete-iteration": 0.1857401077991521, "set_robot_commands": 0.002418480348229706, "deviation-center-line": 1.5947794297543822, "driven_lanedir_consec": 3.537847072747071, "sim_compute_sim_state": 0.010593291623308498, "sim_compute_performance-ego0": 0.002275339272695219}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.714034257011202, "get_ui_image": 0.039584274549742005, "step_physics": 0.1370808897791682, "survival_time": 9.199999999999996, "driven_lanedir": 0.34378355227606616, "get_state_dump": 0.005472883018287453, "get_robot_state": 0.0043805006388071424, "sim_render-ego0": 0.004336505322842984, "get_duckie_state": 1.710170024150127e-06, "in-drivable-lane": 6.300000000000001, "deviation-heading": 2.596887143827367, "agent_compute-ego0": 0.013243740958136482, "complete-iteration": 0.22077808251252048, "set_robot_commands": 0.0026583207620156776, "deviation-center-line": 0.24638239877544305, "driven_lanedir_consec": 0.34378355227606616, "sim_compute_sim_state": 0.011406987422221416, "sim_compute_performance-ego0": 0.0025078812161007445}, "LF-norm-techtrack-000-ego0": {"driven_any": 12.426712929963172, "get_ui_image": 0.034679061864238296, "step_physics": 0.12123102212726423, "survival_time": 59.99999999999873, "driven_lanedir": 1.8676586234219457, "get_state_dump": 0.005388311700558881, "get_robot_state": 0.004158828578126321, "sim_render-ego0": 0.0042201442384997775, "get_duckie_state": 1.6202934576410937e-06, "in-drivable-lane": 49.6999999999991, "deviation-heading": 2.5375325091214247, "agent_compute-ego0": 0.012657076393337074, "complete-iteration": 0.20091775394696976, "set_robot_commands": 0.002454453165783275, "deviation-center-line": 0.9477853992798804, "driven_lanedir_consec": 1.8676586234219457, "sim_compute_sim_state": 0.01364919446489396, "sim_compute_performance-ego0": 0.0023766506919257347}, "LF-norm-small_loop-000-ego0": {"driven_any": 12.010931213701555, "get_ui_image": 0.02813457410401051, "step_physics": 0.11185646136535592, "survival_time": 59.99999999999873, "driven_lanedir": 0.3448755167518305, "get_state_dump": 0.005264863682031433, "get_robot_state": 0.004335819136391671, "sim_render-ego0": 0.004254771708251039, "get_duckie_state": 1.49721309207659e-06, "in-drivable-lane": 57.79999999999873, "deviation-heading": 0.5941546480756017, "agent_compute-ego0": 0.012448818459300376, "complete-iteration": 0.17834183536500955, "set_robot_commands": 0.002512497667666776, "deviation-center-line": 0.14944675462502582, "driven_lanedir_consec": 0.3448755167518305, "sim_compute_sim_state": 0.007014792328770214, "sim_compute_performance-ego0": 0.002421694929454845}}
set_robot_commands_max0.0026583207620156776
set_robot_commands_mean0.0025109379859238586
set_robot_commands_median0.0024834754167250253
set_robot_commands_min0.002418480348229706
sim_compute_performance-ego0_max0.0025078812161007445
sim_compute_performance-ego0_mean0.0023953915275441357
sim_compute_performance-ego0_median0.00239917281069029
sim_compute_performance-ego0_min0.002275339272695219
sim_compute_sim_state_max0.01364919446489396
sim_compute_sim_state_mean0.010666066459798522
sim_compute_sim_state_median0.011000139522764955
sim_compute_sim_state_min0.007014792328770214
sim_render-ego0_max0.004336505322842984
sim_render-ego0_mean0.004226176643826561
sim_render-ego0_median0.004237457973375408
sim_render-ego0_min0.0040932853057124435
simulation-passed1
step_physics_max0.1370808897791682
step_physics_mean0.12129593888654765
step_physics_median0.1181232022008332
step_physics_min0.11185646136535592
survival_time_max59.99999999999873
survival_time_mean47.299999999999045
survival_time_min9.199999999999996
No reset possible
53804LFv-simsuccessyes0:26:48
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
53189LFv-simerrorno0:09:05
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140376672946688
- M:video_aido:cmdline(in:/;out:/) 140376672944336
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
53185LFv-simerrorno0:09:05
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139795326401456
- M:video_aido:cmdline(in:/;out:/) 139795326399920
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
48720LFv-simerrorno0:09:01
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140020184241920
- M:video_aido:cmdline(in:/;out:/) 140020184241008
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
48115LFv-simabortedno0:02:20
KeyboardInterrupt: T [...]
KeyboardInterrupt:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1169, in run_one
    heartbeat()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 628, in heartbeat
    raise KeyboardInterrupt(msg_)
KeyboardInterrupt: The server told us to abort the job because: The challenge has been updated.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
48113LFv-simabortedno0:02:52
KeyboardInterrupt: T [...]
KeyboardInterrupt:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1169, in run_one
    heartbeat()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 628, in heartbeat
    raise KeyboardInterrupt(msg_)
KeyboardInterrupt: The server told us to abort the job because: The challenge has been updated.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
48111LFv-simabortedno0:03:13
KeyboardInterrupt: T [...]
KeyboardInterrupt:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1169, in run_one
    heartbeat()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 628, in heartbeat
    raise KeyboardInterrupt(msg_)
KeyboardInterrupt: The server told us to abort the job because: The challenge has been updated.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43856LFv-simsuccessno0:06:57
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43855LFv-simsuccessno0:08:53
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43854LFv-simsuccessno0:07:54
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible