Duckietown Challenges Home Challenges Submissions

Submission 11651

Submission11651
Competingyes
Challengeaido5-LF-sim-validation
UserBhavya Patwa 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 54151
Next
User labelexercise_state_estimation
Admin priority50
Blessingn/a
User priority50

54151

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
54151LFv-simsuccessyes0:10:31
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.585698939364727
survival_time_median14.775000000000077
deviation-center-line_median0.4912419610220825
in-drivable-lane_median8.650000000000066


other stats
agent_compute-ego0_max0.01405067770200084
agent_compute-ego0_mean0.012982949701165113
agent_compute-ego0_median0.012984626026793674
agent_compute-ego0_min0.011911869049072266
complete-iteration_max0.2135351846426612
complete-iteration_mean0.18976180165862813
complete-iteration_median0.19095370140686913
complete-iteration_min0.163604619178113
deviation-center-line_max0.9201734466840136
deviation-center-line_mean0.5373998422394736
deviation-center-line_min0.2469420002297157
deviation-heading_max5.243820903470425
deviation-heading_mean2.9519095498875574
deviation-heading_median2.7949821226222724
deviation-heading_min0.9738530508352592
driven_any_max3.31399263723808
driven_any_mean2.0775564031943397
driven_any_median1.8203120359800569
driven_any_min1.3556089035791652
driven_lanedir_consec_max0.7738305257319402
driven_lanedir_consec_mean0.5961237436629006
driven_lanedir_consec_min0.4392665701902083
driven_lanedir_max0.7807764461582549
driven_lanedir_mean0.5978602237694792
driven_lanedir_median0.585698939364727
driven_lanedir_min0.4392665701902083
get_duckie_state_max1.5648928555575284e-06
get_duckie_state_mean1.421737837233593e-06
get_duckie_state_median1.4428288601684417e-06
get_duckie_state_min1.2364007730399612e-06
get_robot_state_max0.004130996225451335
get_robot_state_mean0.003884824182007607
get_robot_state_median0.0039553425889333315
get_robot_state_min0.0034976153247124326
get_state_dump_max0.00510370232759773
get_state_dump_mean0.004850810679498449
get_state_dump_median0.004887095593465932
get_state_dump_min0.004525349203464204
get_ui_image_max0.03692604471068872
get_ui_image_mean0.032387115681437054
get_ui_image_median0.032454679046824204
get_ui_image_min0.027713059921411085
in-drivable-lane_max20.150000000000208
in-drivable-lane_mean10.762500000000095
in-drivable-lane_min5.600000000000037
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 2.046023248033159, "get_ui_image": 0.029997513510964132, "step_physics": 0.1166462197448268, "survival_time": 16.4500000000001, "driven_lanedir": 0.7807764461582549, "get_state_dump": 0.005038482492620295, "get_robot_state": 0.0040194569212017636, "sim_render-ego0": 0.00413359800974528, "get_duckie_state": 1.5648928555575284e-06, "in-drivable-lane": 8.050000000000077, "deviation-heading": 5.243820903470425, "agent_compute-ego0": 0.013277068282618668, "complete-iteration": 0.1878304300886212, "set_robot_commands": 0.0024154056202281603, "deviation-center-line": 0.9201734466840136, "driven_lanedir_consec": 0.7738305257319402, "sim_compute_sim_state": 0.009976478056474166, "sim_compute_performance-ego0": 0.0022318760553995767}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.3556089035791652, "get_ui_image": 0.03491184458268427, "step_physics": 0.1216887552126319, "survival_time": 11.250000000000025, "driven_lanedir": 0.5533978052132942, "get_state_dump": 0.004525349203464204, "get_robot_state": 0.0034976153247124326, "sim_render-ego0": 0.003668617358249901, "get_duckie_state": 1.2364007730399612e-06, "in-drivable-lane": 5.600000000000037, "deviation-heading": 2.857560421094429, "agent_compute-ego0": 0.011911869049072266, "complete-iteration": 0.1940769727251171, "set_robot_commands": 0.002122634280044421, "deviation-center-line": 0.49161405731345753, "driven_lanedir_consec": 0.5533978052132942, "sim_compute_sim_state": 0.00977747524734092, "sim_compute_performance-ego0": 0.0018926774506020335}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.5946008239269542, "get_ui_image": 0.03692604471068872, "step_physics": 0.13431070603345285, "survival_time": 13.100000000000051, "driven_lanedir": 0.4392665701902083, "get_state_dump": 0.00510370232759773, "get_robot_state": 0.004130996225451335, "sim_render-ego0": 0.004239180242154082, "get_duckie_state": 1.5338564100374288e-06, "in-drivable-lane": 9.250000000000057, "deviation-heading": 0.9738530508352592, "agent_compute-ego0": 0.01405067770200084, "complete-iteration": 0.2135351846426612, "set_robot_commands": 0.0025274880485389624, "deviation-center-line": 0.2469420002297157, "driven_lanedir_consec": 0.4392665701902083, "sim_compute_sim_state": 0.009878930936748085, "sim_compute_performance-ego0": 0.0022706169592563645}, "LF-norm-small_loop-000-ego0": {"driven_any": 3.31399263723808, "get_ui_image": 0.027713059921411085, "step_physics": 0.10031179823481876, "survival_time": 26.00000000000023, "driven_lanedir": 0.6180000735161597, "get_state_dump": 0.00473570869431157, "get_robot_state": 0.0038912282566648986, "sim_render-ego0": 0.003919062367327611, "get_duckie_state": 1.351801310299454e-06, "in-drivable-lane": 20.150000000000208, "deviation-heading": 2.7324038241501154, "agent_compute-ego0": 0.01269218377096868, "complete-iteration": 0.163604619178113, "set_robot_commands": 0.002341604049741192, "deviation-center-line": 0.4908698647307074, "driven_lanedir_consec": 0.6180000735161597, "sim_compute_sim_state": 0.005824126574906187, "sim_compute_performance-ego0": 0.0020929055570869663}}
set_robot_commands_max0.0025274880485389624
set_robot_commands_mean0.002351782999638184
set_robot_commands_median0.002378504834984676
set_robot_commands_min0.002122634280044421
sim_compute_performance-ego0_max0.0022706169592563645
sim_compute_performance-ego0_mean0.002122019005586235
sim_compute_performance-ego0_median0.0021623908062432715
sim_compute_performance-ego0_min0.0018926774506020335
sim_compute_sim_state_max0.009976478056474166
sim_compute_sim_state_mean0.008864252703867339
sim_compute_sim_state_median0.0098282030920445
sim_compute_sim_state_min0.005824126574906187
sim_render-ego0_max0.004239180242154082
sim_render-ego0_mean0.003990114494369219
sim_render-ego0_median0.004026330188536445
sim_render-ego0_min0.003668617358249901
simulation-passed1
step_physics_max0.13431070603345285
step_physics_mean0.11823936980643258
step_physics_median0.11916748747872936
step_physics_min0.10031179823481876
survival_time_max26.00000000000023
survival_time_mean16.700000000000102
survival_time_min11.250000000000025
No reset possible
54148LFv-simsuccessyes0:12:38
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
54146LFv-simsuccessyes0:07:54
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
54142LFv-simsuccessyes0:12:59
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
54140LFv-simsuccessyes0:08:27
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49232LFv-simerrorno0:06:02
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140210063720848
- M:video_aido:cmdline(in:/;out:/) 140210063721088
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49223LFv-simhost-errorno0:01:17
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 83, in run_docker
    return subprocess.check_output(cmd0, cwd=cwd, stderr=sys.stderr)
  File "/usr/lib/python3.8/subprocess.py", line 411, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/lib/python3.8/subprocess.py", line 512, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['docker-compose', '-p', 'nogpu-prod-05_a352a02a43af-job49223-674561', 'ps', '-q', 'evaluator', 'simulator', 'solution-ego0']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 42, in get_services_id
    output = run_docker(wd, project, cmd, get_output=True)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 90, in run_docker
    raise DockerComposeFail(
duckietown_challenges_runner.docker_compose.DockerComposeFail: Could not run command
│    cmd: [docker-compose, -p, nogpu-prod-05_a352a02a43af-job49223-674561, ps, -q, evaluator,
│          simulator, solution-ego0]
│ stdout: ''
│  sderr: ''
│      e: Command '['docker-compose', '-p', 'nogpu-prod-05_a352a02a43af-job49223-674561', 'ps', '-q', 'evaluator', 'simulator', 'solution-ego0']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 777, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 991, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 120, in write_logs
    services2id: Dict[ServiceName, ContainerID] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 63, in get_services_id
    raise DockerComposeFail(msg, output=output.decode(), names=names) from e
duckietown_challenges_runner.docker_compose.DockerComposeFail: Cannot get process ids
│ output: not available
│  names: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43265LFv-simsuccessno0:06:35
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43264LFv-simhost-errorno0:00:17
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 687, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 888, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1388, in write_logs
    services2id: Dict[str, str] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 937, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43263LFv-simsuccessno0:11:55
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43262LFv-simsuccessno0:07:02
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible