Duckietown Challenges Home Challenges Submissions

Submission 11701

Submission11701
Competingyes
Challengeaido5-LF-sim-validation
UserYishu Malhotra 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 53952
Next
User labelreal-exercise-2
Admin priority50
Blessingn/a
User priority50

53952

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
53952LFv-simsuccessyes0:41:03
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median9.953866111566352
survival_time_median59.99999999999873
deviation-center-line_median3.5751955038466185
in-drivable-lane_median1.1249999999999662


other stats
agent_compute-ego0_max0.013237876360065038
agent_compute-ego0_mean0.012941119027673751
agent_compute-ego0_median0.01288147314105006
agent_compute-ego0_min0.012763653468529848
complete-iteration_max0.25953710525855733
complete-iteration_mean0.23206583263475036
complete-iteration_median0.236881963914876
complete-iteration_min0.19496229745069213
deviation-center-line_max3.890078329783319
deviation-center-line_mean3.5034521329355828
deviation-center-line_min2.973339194265777
deviation-heading_max19.064864413688102
deviation-heading_mean18.38839204383128
deviation-heading_median18.890663517648164
deviation-heading_min16.707376726340698
driven_any_max11.646172315979967
driven_any_mean11.011778982654086
driven_any_median11.008189266762791
driven_any_min10.384565081110791
driven_lanedir_consec_max10.804361064633682
driven_lanedir_consec_mean8.790537106862601
driven_lanedir_consec_min4.4500551396840216
driven_lanedir_max10.804361064633682
driven_lanedir_mean9.418845114439744
driven_lanedir_median9.953866111566352
driven_lanedir_min6.9632871699925865
get_duckie_state_max1.5516066729873545e-06
get_duckie_state_mean1.472150356346721e-06
get_duckie_state_median1.4564179063935164e-06
get_duckie_state_min1.4241589396124974e-06
get_robot_state_max0.004070259748549386
get_robot_state_mean0.003833898944124195
get_robot_state_median0.0037741350393112657
get_robot_state_min0.003717065949324863
get_state_dump_max0.005129982688643355
get_state_dump_mean0.004807327460686829
get_state_dump_median0.004726299139780367
get_state_dump_min0.004646728874543227
get_ui_image_max0.03647862008767362
get_ui_image_mean0.03176630590480134
get_ui_image_median0.03205340103146238
get_ui_image_min0.02647980146860699
in-drivable-lane_max16.300000000000036
in-drivable-lane_mean4.637499999999992
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 11.227310662176505, "get_ui_image": 0.029725630019328476, "step_physics": 0.14884993634950508, "survival_time": 59.99999999999873, "driven_lanedir": 9.930862200080655, "get_state_dump": 0.00474437110926289, "get_robot_state": 0.003800077501879842, "sim_render-ego0": 0.003847743946745632, "get_duckie_state": 1.48331692177092e-06, "in-drivable-lane": 2.2499999999999325, "deviation-heading": 19.064864413688102, "agent_compute-ego0": 0.012931678614747416, "complete-iteration": 0.21809231470665463, "set_robot_commands": 0.002302996423421156, "deviation-center-line": 3.5172518473077172, "driven_lanedir_consec": 9.930862200080655, "sim_compute_sim_state": 0.009720872383530591, "sim_compute_performance-ego0": 0.0020766812101391135}, "LF-norm-zigzag-000-ego0": {"driven_any": 11.646172315979967, "get_ui_image": 0.03647862008767362, "step_physics": 0.17561493388421331, "survival_time": 59.99999999999873, "driven_lanedir": 10.804361064633682, "get_state_dump": 0.004646728874543227, "get_robot_state": 0.003717065949324863, "sim_render-ego0": 0.0038376997948486938, "get_duckie_state": 1.4241589396124974e-06, "in-drivable-lane": 0.0, "deviation-heading": 18.758066330267702, "agent_compute-ego0": 0.012831267667352706, "complete-iteration": 0.2556716131230973, "set_robot_commands": 0.0022293530733361036, "deviation-center-line": 3.890078329783319, "driven_lanedir_consec": 10.804361064633682, "sim_compute_sim_state": 0.01416438981754198, "sim_compute_performance-ego0": 0.002061234227227331}, "LF-norm-techtrack-000-ego0": {"driven_any": 10.78906787134908, "get_ui_image": 0.03438117204359628, "step_physics": 0.17955330170560738, "survival_time": 59.99999999999873, "driven_lanedir": 9.97687002305205, "get_state_dump": 0.005129982688643355, "get_robot_state": 0.004070259748549386, "sim_render-ego0": 0.004148917233914162, "get_duckie_state": 1.5516066729873545e-06, "in-drivable-lane": 0.0, "deviation-heading": 19.02326070502863, "agent_compute-ego0": 0.013237876360065038, "complete-iteration": 0.25953710525855733, "set_robot_commands": 0.002496432900726547, "deviation-center-line": 3.63313916038552, "driven_lanedir_consec": 9.97687002305205, "sim_compute_sim_state": 0.014080807529420876, "sim_compute_performance-ego0": 0.0023317140504581346}, "LF-norm-small_loop-000-ego0": {"driven_any": 10.384565081110791, "get_ui_image": 0.02647980146860699, "step_physics": 0.13278739676685952, "survival_time": 59.99999999999873, "driven_lanedir": 6.9632871699925865, "get_state_dump": 0.004708227170297843, "get_robot_state": 0.003748192576742688, "sim_render-ego0": 0.003869212140250861, "get_duckie_state": 1.429518891016113e-06, "in-drivable-lane": 16.300000000000036, "deviation-heading": 16.707376726340698, "agent_compute-ego0": 0.012763653468529848, "complete-iteration": 0.19496229745069213, "set_robot_commands": 0.002245242550013762, "deviation-center-line": 2.973339194265777, "driven_lanedir_consec": 4.4500551396840216, "sim_compute_sim_state": 0.0062303912331917, "sim_compute_performance-ego0": 0.0020418429156326434}}
set_robot_commands_max0.002496432900726547
set_robot_commands_mean0.002318506236874392
set_robot_commands_median0.002274119486717459
set_robot_commands_min0.0022293530733361036
sim_compute_performance-ego0_max0.0023317140504581346
sim_compute_performance-ego0_mean0.0021278681008643054
sim_compute_performance-ego0_median0.002068957718683222
sim_compute_performance-ego0_min0.0020418429156326434
sim_compute_sim_state_max0.01416438981754198
sim_compute_sim_state_mean0.011049115240921288
sim_compute_sim_state_median0.011900839956475734
sim_compute_sim_state_min0.0062303912331917
sim_render-ego0_max0.004148917233914162
sim_render-ego0_mean0.003925893278939837
sim_render-ego0_median0.0038584780434982465
sim_render-ego0_min0.0038376997948486938
simulation-passed1
step_physics_max0.17955330170560738
step_physics_mean0.15920139217654633
step_physics_median0.1622324351168592
step_physics_min0.13278739676685952
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
48905LFv-simerrorno0:09:36
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140509253142416
- M:video_aido:cmdline(in:/;out:/) 140508871580400
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
48886LFv-simhost-errorno0:02:39
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 421, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 416, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1347, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 719, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 400, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 735, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 665, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 423, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 330, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1178, in run_one
    stats = container.stats(stream=False)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 418, in stats
    return self.client.api.stats(self.id, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 1125, in stats
    return self._result(self._get(url, params={'stream': False}),
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 230, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 543, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43639LFv-simsuccessno0:10:30
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43638LFv-simhost-errorno0:00:13
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 687, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 888, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1388, in write_logs
    services2id: Dict[str, str] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 937, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43637LFv-simsuccessno0:08:09
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible