Duckietown Challenges Home Challenges Submissions

Submission 11745

Submission11745
Competingyes
Challengeaido5-LF-sim-validation
UserJean-Sébastien Grondin 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 53862
Next
User labelsim-exercise-2
Admin priority50
Blessingn/a
User priority50

53862

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
53862LFv-simsuccessyes0:33:14
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median5.6694150188764265
survival_time_median59.99999999999873
deviation-center-line_median3.05176049147522
in-drivable-lane_median16.824999999999378


other stats
agent_compute-ego0_max0.01361472739665137
agent_compute-ego0_mean0.013109186927987232
agent_compute-ego0_median0.013065573774110663
agent_compute-ego0_min0.012690872767076231
complete-iteration_max0.21963564541615813
complete-iteration_mean0.18829105823146117
complete-iteration_median0.1841315164590041
complete-iteration_min0.16526555459167835
deviation-center-line_max3.972915213865975
deviation-center-line_mean3.141725765322417
deviation-center-line_min2.490466864473253
deviation-heading_max20.81534559430188
deviation-heading_mean15.122770080320093
deviation-heading_median15.405068245872704
deviation-heading_min8.86559823523308
driven_any_max12.383893246825522
driven_any_mean9.727319828716391
driven_any_median9.239189269478771
driven_any_min8.047007529082496
driven_lanedir_consec_max7.312977281876021
driven_lanedir_consec_mean5.558994328841678
driven_lanedir_consec_min3.5841699957378363
driven_lanedir_max9.859232868729997
driven_lanedir_mean6.944636976763896
driven_lanedir_median6.836871703987441
driven_lanedir_min4.245571630350703
get_duckie_state_max1.3276798143474189e-06
get_duckie_state_mean1.2940546616313322e-06
get_duckie_state_median1.3038054119408005e-06
get_duckie_state_min1.240928008296309e-06
get_robot_state_max0.00387317354137157
get_robot_state_mean0.003783083252463973
get_robot_state_median0.003775637711613105
get_robot_state_min0.0037078840452581113
get_state_dump_max0.005213249533698521
get_state_dump_mean0.004810498086354253
get_state_dump_median0.004694870866208549
get_state_dump_min0.004639001079301393
get_ui_image_max0.035686326562911645
get_ui_image_mean0.030543395601454287
get_ui_image_median0.029787651267698587
get_ui_image_min0.02691195330750833
in-drivable-lane_max29.149999999999697
in-drivable-lane_mean18.29999999999955
in-drivable-lane_min10.399999999999752
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 9.94970472796326, "get_ui_image": 0.028231170751173033, "step_physics": 0.11130736034974568, "survival_time": 59.99999999999873, "driven_lanedir": 7.066757961521271, "get_state_dump": 0.004711656348095845, "get_robot_state": 0.003759003003173625, "sim_render-ego0": 0.003865740479875067, "get_duckie_state": 1.3276798143474189e-06, "in-drivable-lane": 20.649999999999125, "deviation-heading": 16.253594321599856, "agent_compute-ego0": 0.012968658507614708, "complete-iteration": 0.17898225089493242, "set_robot_commands": 0.0022890508224525418, "deviation-center-line": 3.4020501530081613, "driven_lanedir_consec": 5.977708599065014, "sim_compute_sim_state": 0.009679749248228304, "sim_compute_performance-ego0": 0.0020771318430904545}, "LF-norm-zigzag-000-ego0": {"driven_any": 8.528673810994286, "get_ui_image": 0.035686326562911645, "step_physics": 0.14107459590000276, "survival_time": 59.99999999999873, "driven_lanedir": 4.245571630350703, "get_state_dump": 0.004678085384321252, "get_robot_state": 0.003792272420052585, "sim_render-ego0": 0.0039469707419135785, "get_duckie_state": 1.3046518749837376e-06, "in-drivable-lane": 29.149999999999697, "deviation-heading": 20.81534559430188, "agent_compute-ego0": 0.01361472739665137, "complete-iteration": 0.21963564541615813, "set_robot_commands": 0.002282401504961279, "deviation-center-line": 2.7014708299422785, "driven_lanedir_consec": 3.5841699957378363, "sim_compute_sim_state": 0.012329305240653338, "sim_compute_performance-ego0": 0.0021386011554041474}, "LF-norm-techtrack-000-ego0": {"driven_any": 8.047007529082496, "get_ui_image": 0.03134413178422414, "step_physics": 0.11723096939168366, "survival_time": 42.099999999999746, "driven_lanedir": 6.606985446453613, "get_state_dump": 0.004639001079301393, "get_robot_state": 0.0037078840452581113, "sim_render-ego0": 0.003779371720898901, "get_duckie_state": 1.302958948897863e-06, "in-drivable-lane": 10.399999999999752, "deviation-heading": 8.86559823523308, "agent_compute-ego0": 0.012690872767076231, "complete-iteration": 0.18928078202307577, "set_robot_commands": 0.0022293726603190103, "deviation-center-line": 2.490466864473253, "driven_lanedir_consec": 5.36112143868784, "sim_compute_sim_state": 0.011542611274854992, "sim_compute_performance-ego0": 0.0020231270139610498}, "LF-norm-small_loop-000-ego0": {"driven_any": 12.383893246825522, "get_ui_image": 0.02691195330750833, "step_physics": 0.10100901652930876, "survival_time": 59.99999999999873, "driven_lanedir": 9.859232868729997, "get_state_dump": 0.005213249533698521, "get_robot_state": 0.00387317354137157, "sim_render-ego0": 0.003956611706354933, "get_duckie_state": 1.240928008296309e-06, "in-drivable-lane": 12.999999999999629, "deviation-heading": 14.556542170145557, "agent_compute-ego0": 0.013162489040606623, "complete-iteration": 0.16526555459167835, "set_robot_commands": 0.0024470278464387993, "deviation-center-line": 3.972915213865975, "driven_lanedir_consec": 7.312977281876021, "sim_compute_sim_state": 0.006393656742562859, "sim_compute_performance-ego0": 0.0022084558139137183}}
set_robot_commands_max0.0024470278464387993
set_robot_commands_mean0.002311963208542908
set_robot_commands_median0.00228572616370691
set_robot_commands_min0.0022293726603190103
sim_compute_performance-ego0_max0.0022084558139137183
sim_compute_performance-ego0_mean0.0021118289565923423
sim_compute_performance-ego0_median0.002107866499247301
sim_compute_performance-ego0_min0.0020231270139610498
sim_compute_sim_state_max0.012329305240653338
sim_compute_sim_state_mean0.009986330626574872
sim_compute_sim_state_median0.010611180261541647
sim_compute_sim_state_min0.006393656742562859
sim_render-ego0_max0.003956611706354933
sim_render-ego0_mean0.00388717366226062
sim_render-ego0_median0.003906355610894323
sim_render-ego0_min0.003779371720898901
simulation-passed1
step_physics_max0.14107459590000276
step_physics_mean0.11765548554268522
step_physics_median0.11426916487071467
step_physics_min0.10100901652930876
survival_time_max59.99999999999873
survival_time_mean55.52499999999898
survival_time_min42.099999999999746
No reset possible
53860LFv-simsuccessyes0:30:26
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
53857LFv-simsuccessyes0:30:56
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
53854LFv-simhost-erroryes0:34:58
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 421, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 416, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1347, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 719, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 400, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 735, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 665, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 423, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 330, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1176, in run_one
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 758, in inspect_container
    self._get(self._url("/containers/{0}/json", container)), True
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 230, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 543, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
53852LFv-simhost-erroryes0:30:38
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 421, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 416, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1347, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 719, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 400, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/packages/six.py", line 735, in reraise
    raise value
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 665, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 423, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 330, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1176, in run_one
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 758, in inspect_container
    self._get(self._url("/containers/{0}/json", container)), True
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 230, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 543, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
48193LFv-simerrorno0:08:42
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139720272327968
- M:video_aido:cmdline(in:/;out:/) 139720272329648
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
48170LFv-simabortedno0:00:30
KeyboardInterrupt: T [...]
KeyboardInterrupt:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1169, in run_one
    heartbeat()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 628, in heartbeat
    raise KeyboardInterrupt(msg_)
KeyboardInterrupt: The server told us to abort the job because: The challenge has been updated.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
48144LFv-simabortedno0:00:39
KeyboardInterrupt: T [...]
KeyboardInterrupt:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1169, in run_one
    heartbeat()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 628, in heartbeat
    raise KeyboardInterrupt(msg_)
KeyboardInterrupt: The server told us to abort the job because: The challenge has been updated.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43797LFv-simsuccessno0:07:33
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43796LFv-simsuccessno0:11:20
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43795LFv-simsuccessno0:08:04
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43794LFv-simsuccessno0:07:33
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible