Duckietown Challenges Home Challenges Submissions

Submission 11359

Submission11359
Competingyes
Challengeaido5-LF-sim-validation
UserYishu Malhotra 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 54823
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

54823

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
54823LFv-simsuccessyes0:14:57
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.1646585231460795
survival_time_median23.850000000000204
deviation-center-line_median0.4227641434875059
in-drivable-lane_median17.700000000000195


other stats
agent_compute-ego0_max0.012382181421402962
agent_compute-ego0_mean0.012057194622628886
agent_compute-ego0_median0.012050367848877176
agent_compute-ego0_min0.011745861371358236
complete-iteration_max0.1984395283845163
complete-iteration_mean0.17829791814045903
complete-iteration_median0.18191807544036903
complete-iteration_min0.15091599329658176
deviation-center-line_max0.8178865303951087
deviation-center-line_mean0.45274466424129234
deviation-center-line_min0.14756383959504887
deviation-heading_max3.5697402640870433
deviation-heading_mean2.179015810355746
deviation-heading_median2.190388457184699
deviation-heading_min0.7655460629665434
driven_any_max11.585098325155236
driven_any_mean5.627211885833123
driven_any_median4.794574346866308
driven_any_min1.3346005244446413
driven_lanedir_consec_max3.629497534636787
driven_lanedir_consec_mean1.6510450816652893
driven_lanedir_consec_min0.6453657457322111
driven_lanedir_max3.629497534636787
driven_lanedir_mean1.727136022560568
driven_lanedir_median1.3156550619214675
driven_lanedir_min0.647736431762548
get_duckie_state_max1.2000401814778646e-06
get_duckie_state_mean1.1390213645952965e-06
get_duckie_state_median1.1358049608046008e-06
get_duckie_state_min1.08443535529412e-06
get_robot_state_max0.0035178305641297373
get_robot_state_mean0.003501986086496114
get_robot_state_median0.003512089939225407
get_robot_state_min0.003465933903403904
get_state_dump_max0.004396897169851487
get_state_dump_mean0.004342868158264425
get_state_dump_median0.0043328498579286315
get_state_dump_min0.004308875747348951
get_ui_image_max0.03569037119547526
get_ui_image_mean0.030493066413445385
get_ui_image_median0.03038835362892147
get_ui_image_min0.025505187200463336
in-drivable-lane_max34.89999999999967
in-drivable-lane_mean18.587500000000013
in-drivable-lane_min4.049999999999986
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 11.585098325155236, "get_ui_image": 0.02805374814318372, "step_physics": 0.1082032891539308, "survival_time": 49.9999999999993, "driven_lanedir": 3.629497534636787, "get_state_dump": 0.004348282571081872, "get_robot_state": 0.00351013479890166, "sim_render-ego0": 0.003624836762587389, "get_duckie_state": 1.08443535529412e-06, "in-drivable-lane": 34.89999999999967, "deviation-heading": 3.5697402640870433, "agent_compute-ego0": 0.011841585109760236, "complete-iteration": 0.1725085615278124, "set_robot_commands": 0.002045116939030208, "deviation-center-line": 0.8178865303951087, "driven_lanedir_consec": 3.629497534636787, "sim_compute_sim_state": 0.008951102817927922, "sim_compute_performance-ego0": 0.0018598707048566668}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.3346005244446413, "get_ui_image": 0.03569037119547526, "step_physics": 0.1189354960123698, "survival_time": 7.449999999999981, "driven_lanedir": 0.647736431762548, "get_state_dump": 0.004317417144775391, "get_robot_state": 0.0035140450795491537, "sim_render-ego0": 0.003635981877644857, "get_duckie_state": 1.2000401814778646e-06, "in-drivable-lane": 4.049999999999986, "deviation-heading": 1.1802650112754056, "agent_compute-ego0": 0.011745861371358236, "complete-iteration": 0.19132758935292565, "set_robot_commands": 0.002035228411356608, "deviation-center-line": 0.14756383959504887, "driven_lanedir_consec": 0.6453657457322111, "sim_compute_sim_state": 0.009505565961201983, "sim_compute_performance-ego0": 0.0018763415018717448}, "LF-norm-techtrack-000-ego0": {"driven_any": 4.6300143892584815, "get_ui_image": 0.032722959114659216, "step_physics": 0.12755578756332395, "survival_time": 24.750000000000217, "driven_lanedir": 1.9148061314168887, "get_state_dump": 0.004396897169851487, "get_robot_state": 0.0035178305641297373, "sim_render-ego0": 0.003716359696080608, "get_duckie_state": 1.1272007419217017e-06, "in-drivable-lane": 15.550000000000194, "deviation-heading": 3.200511903093992, "agent_compute-ego0": 0.012382181421402962, "complete-iteration": 0.1984395283845163, "set_robot_commands": 0.0021017097657726655, "deviation-center-line": 0.6864666915388863, "driven_lanedir_consec": 1.612813053866112, "sim_compute_sim_state": 0.010078057166068783, "sim_compute_performance-ego0": 0.00189477349481275}, "LF-norm-small_loop-000-ego0": {"driven_any": 4.9591343044741345, "get_ui_image": 0.025505187200463336, "step_physics": 0.0913339350534522, "survival_time": 22.95000000000019, "driven_lanedir": 0.7165039924260468, "get_state_dump": 0.004308875747348951, "get_robot_state": 0.003465933903403904, "sim_render-ego0": 0.0037028219388878865, "get_duckie_state": 1.1444091796875e-06, "in-drivable-lane": 19.850000000000193, "deviation-heading": 0.7655460629665434, "agent_compute-ego0": 0.01225915058799412, "complete-iteration": 0.15091599329658176, "set_robot_commands": 0.002132328696872877, "deviation-center-line": 0.15906159543612547, "driven_lanedir_consec": 0.7165039924260468, "sim_compute_sim_state": 0.00625859084336654, "sim_compute_performance-ego0": 0.001880129523899244}}
set_robot_commands_max0.002132328696872877
set_robot_commands_mean0.0020785959532580896
set_robot_commands_median0.0020734133524014367
set_robot_commands_min0.002035228411356608
sim_compute_performance-ego0_max0.00189477349481275
sim_compute_performance-ego0_mean0.0018777788063601016
sim_compute_performance-ego0_median0.0018782355128854943
sim_compute_performance-ego0_min0.0018598707048566668
sim_compute_sim_state_max0.010078057166068783
sim_compute_sim_state_mean0.008698329197141308
sim_compute_sim_state_median0.009228334389564954
sim_compute_sim_state_min0.00625859084336654
sim_render-ego0_max0.003716359696080608
sim_render-ego0_mean0.003670000068800185
sim_render-ego0_median0.003669401908266372
sim_render-ego0_min0.003624836762587389
simulation-passed1
step_physics_max0.12755578756332395
step_physics_mean0.1115071269457692
step_physics_median0.1135693925831503
step_physics_min0.0913339350534522
survival_time_max49.9999999999993
survival_time_mean26.287499999999923
survival_time_min7.449999999999981
No reset possible
54815LFv-simsuccessyes0:15:02
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
54810LFv-simsuccessyes0:22:18
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
54808LFv-simsuccessyes0:27:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
54800LFv-simsuccessyes0:23:04
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49957LFv-simerrorno0:04:51
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140433309632688
- M:video_aido:cmdline(in:/;out:/) 140434135573168
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49945LFv-simerrorno0:09:48
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140542814242272
- M:video_aido:cmdline(in:/;out:/) 140542814243376
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42043LFv-simsuccessno0:09:33
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42042LFv-simsuccessno0:10:38
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42041LFv-simhost-errorno0:00:10
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 687, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 888, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1388, in write_logs
    services2id: Dict[str, str] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 937, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42040LFv-simhost-errorno0:00:10
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 687, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 888, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1388, in write_logs
    services2id: Dict[str, str] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 937, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42036LFv-simhost-errorno0:00:10
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 687, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 888, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1388, in write_logs
    services2id: Dict[str, str] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 937, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42035LFv-simhost-errorno0:00:10
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 687, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 888, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1388, in write_logs
    services2id: Dict[str, str] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 937, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42034LFv-simhost-errorno0:00:10
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 687, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 888, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1388, in write_logs
    services2id: Dict[str, str] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 937, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible