Duckietown Challenges Home Challenges Submissions

Submission 11659

Submission11659
Competingyes
Challengeaido5-LF-sim-validation
UserBhavya Patwa 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 54116
Next
User labelexercise_state_estimation
Admin priority50
Blessingn/a
User priority50

54116

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
54116LFv-simsuccessyes0:06:46
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.5947041908916109
survival_time_median6.8999999999999835
deviation-center-line_median0.10758083999525164
in-drivable-lane_median4.424999999999985


other stats
agent_compute-ego0_max0.013399598210356956
agent_compute-ego0_mean0.013017942306416523
agent_compute-ego0_median0.012975934916926968
agent_compute-ego0_min0.0127203011814552
complete-iteration_max0.2269706606864929
complete-iteration_mean0.2019415181835287
complete-iteration_median0.20430659278239807
complete-iteration_min0.17218222648282594
deviation-center-line_max0.17622539165075146
deviation-center-line_mean0.11681967269485648
deviation-center-line_min0.07589161913817119
deviation-heading_max1.3028686127086373
deviation-heading_mean0.7029260276916234
deviation-heading_median0.5715156736426281
deviation-heading_min0.3658041507726006
driven_any_max3.774268132554616
driven_any_mean1.9785683348232812
driven_any_median1.5376634489022607
driven_any_min1.064678308933987
driven_lanedir_consec_max0.664465795453598
driven_lanedir_consec_mean0.5270237442848184
driven_lanedir_consec_min0.2542207999024536
driven_lanedir_max0.664465795453598
driven_lanedir_mean0.5270237442848184
driven_lanedir_median0.5947041908916109
driven_lanedir_min0.2542207999024536
get_duckie_state_max1.4970468920330668e-06
get_duckie_state_mean1.3896128927637137e-06
get_duckie_state_median1.3720710147428272e-06
get_duckie_state_min1.317262649536133e-06
get_robot_state_max0.003845935644105423
get_robot_state_mean0.0038113951807613447
get_robot_state_median0.003800242699148283
get_robot_state_min0.003799159680643389
get_state_dump_max0.005142048347828
get_state_dump_mean0.004894996626753856
get_state_dump_median0.004842206707564733
get_state_dump_min0.004753524744057957
get_ui_image_max0.03700392047564189
get_ui_image_mean0.03238193334206166
get_ui_image_median0.03252014914999368
get_ui_image_min0.027483514592617372
in-drivable-lane_max16.800000000000118
in-drivable-lane_mean6.912500000000019
in-drivable-lane_min1.999999999999993
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 3.774268132554616, "get_ui_image": 0.029594511114140992, "step_physics": 0.12507342138598043, "survival_time": 18.55000000000013, "driven_lanedir": 0.2542207999024536, "get_state_dump": 0.004810817780033234, "get_robot_state": 0.003799159680643389, "sim_render-ego0": 0.003887382245832874, "get_duckie_state": 1.3875704939647388e-06, "in-drivable-lane": 16.800000000000118, "deviation-heading": 1.3028686127086373, "agent_compute-ego0": 0.01298440848627398, "complete-iteration": 0.19343150495201028, "set_robot_commands": 0.002267199818805982, "deviation-center-line": 0.17622539165075146, "driven_lanedir_consec": 0.2542207999024536, "sim_compute_sim_state": 0.008909511309798045, "sim_compute_performance-ego0": 0.0020160380230155044}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.280073118172269, "get_ui_image": 0.03700392047564189, "step_physics": 0.14974052707354227, "survival_time": 5.949999999999987, "driven_lanedir": 0.5601980645939806, "get_state_dump": 0.004873595635096232, "get_robot_state": 0.0038004418214162192, "sim_render-ego0": 0.003884655237197876, "get_duckie_state": 1.317262649536133e-06, "in-drivable-lane": 3.349999999999988, "deviation-heading": 0.7582930187385377, "agent_compute-ego0": 0.012967461347579956, "complete-iteration": 0.2269706606864929, "set_robot_commands": 0.0022989173730214437, "deviation-center-line": 0.09989380909988084, "driven_lanedir_consec": 0.5601980645939806, "sim_compute_sim_state": 0.010232688983281451, "sim_compute_performance-ego0": 0.0020783722400665285}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.064678308933987, "get_ui_image": 0.03544578718584637, "step_physics": 0.13998678395914477, "survival_time": 4.249999999999993, "driven_lanedir": 0.6292103171892413, "get_state_dump": 0.005142048347828, "get_robot_state": 0.003845935644105423, "sim_render-ego0": 0.00409139588821766, "get_duckie_state": 1.4970468920330668e-06, "in-drivable-lane": 1.999999999999993, "deviation-heading": 0.3847383285467184, "agent_compute-ego0": 0.013399598210356956, "complete-iteration": 0.21518168061278584, "set_robot_commands": 0.00242492210033328, "deviation-center-line": 0.07589161913817119, "driven_lanedir_consec": 0.6292103171892413, "sim_compute_sim_state": 0.00858160506847293, "sim_compute_performance-ego0": 0.002161982447602028}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.7952537796322523, "get_ui_image": 0.027483514592617372, "step_physics": 0.11018302018129372, "survival_time": 7.84999999999998, "driven_lanedir": 0.664465795453598, "get_state_dump": 0.004753524744057957, "get_robot_state": 0.0038000435768803464, "sim_render-ego0": 0.003875337069547629, "get_duckie_state": 1.3565715355209158e-06, "in-drivable-lane": 5.4999999999999805, "deviation-heading": 0.3658041507726006, "agent_compute-ego0": 0.0127203011814552, "complete-iteration": 0.17218222648282594, "set_robot_commands": 0.002294409124157097, "deviation-center-line": 0.1152678708906224, "driven_lanedir_consec": 0.664465795453598, "sim_compute_sim_state": 0.005024446716791467, "sim_compute_performance-ego0": 0.001960353006290484}}
set_robot_commands_max0.00242492210033328
set_robot_commands_mean0.0023213621040794503
set_robot_commands_median0.0022966632485892703
set_robot_commands_min0.002267199818805982
sim_compute_performance-ego0_max0.002161982447602028
sim_compute_performance-ego0_mean0.002054186429243636
sim_compute_performance-ego0_median0.0020472051315410167
sim_compute_performance-ego0_min0.001960353006290484
sim_compute_sim_state_max0.010232688983281451
sim_compute_sim_state_mean0.008187063019585973
sim_compute_sim_state_median0.008745558189135487
sim_compute_sim_state_min0.005024446716791467
sim_render-ego0_max0.00409139588821766
sim_render-ego0_mean0.003934692610199009
sim_render-ego0_median0.003886018741515375
sim_render-ego0_min0.003875337069547629
simulation-passed1
step_physics_max0.14974052707354227
step_physics_mean0.1312459381499903
step_physics_median0.1325301026725626
step_physics_min0.11018302018129372
survival_time_max18.55000000000013
survival_time_mean9.150000000000022
survival_time_min4.249999999999993
No reset possible
54111LFv-simsuccessyes0:09:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49210LFv-simerrorno0:03:25
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140416322610032
- M:video_aido:cmdline(in:/;out:/) 140416322611376
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49187LFv-simerrorno0:03:15
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139818357421440
- M:video_aido:cmdline(in:/;out:/) 139818357512944
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49180LFv-simerrorno0:01:39
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139708843508496
- M:video_aido:cmdline(in:/;out:/) 139708843989936
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43321LFv-simsuccessno0:05:38
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43320LFv-simsuccessno0:07:11
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43317LFv-simfailedno0:02:47
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43316LFv-simsuccessno0:08:05
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43315LFv-simsuccessno0:05:01
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43313LFv-simhost-errorno0:00:32
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 687, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 888, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1388, in write_logs
    services2id: Dict[str, str] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 937, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43310LFv-simhost-errorno0:00:40
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 687, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 888, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1388, in write_logs
    services2id: Dict[str, str] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 937, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43308LFv-simhost-errorno0:00:48
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 687, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 888, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1388, in write_logs
    services2id: Dict[str, str] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 937, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible