Duckietown Challenges Home Challenges Submissions

Evaluator 5347

ID5347
evaluatornogpu-production-b-spot-0-02
ownerI don't have one 😀
machinenogpu-production-b-spot-0_0e7d0515eb85
processnogpu-production-b-spot-0-02_0e7d0515eb85
version6.2.7
first heard
last heard
statusinactive
# evaluating
# success4 79653
# timeout
# failed
# error
# aborted3 79722
# host-error
arm0
x86_641
Mac0
gpu available0
Number of processors96
Processor frequency (MHz)0.0 GHz
Free % of processors100%
RAM total (MB)184.7 GB
RAM free (MB)181.9 GB
Disk (MB)969.3 GB
Disk available (MB)954.4 GB
Docker Hub
P11
P2
Cloud simulations1
PI Camera0
# Duckiebots0
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
7972413798Nicholas Kostelniktemplate-randomaido-hello-sim-validation370abortednonogpu-production-b-spot-0-020:00:22
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 261, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 941, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.35/images/create?tag=sha256%3Ab13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691&fromImage=docker.io%2Fnitaigao%2Faido-submissions

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 65, in docker_pull
    pulling = client.api.pull(repository=repository, tag=br.tag, stream=True, decode=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/image.py", line 415, in pull
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 263, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.ImageNotFound: 404 Client Error: Not Found ("pull access denied for nitaigao/aido-submissions, repository does not exist or may require 'docker login': denied: requested access to the resource is denied")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 38, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    raise PullError(msg) from e
duckietown_build_utils.docker_pulling.PullError: Cannot pull repo  docker.io/nitaigao/aido-submissions@sha256:b13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691  tag  None

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 745, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 944, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 42, in docker_pull_retry
    raise PullError(msg) from e
duckietown_build_utils.docker_pulling.PullError: After trying 4 I still could not pull docker.io/nitaigao/aido-submissions@sha256:b13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
7972313798Nicholas Kostelniktemplate-randomaido-hello-sim-validation370abortednonogpu-production-b-spot-0-020:00:22
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 261, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 941, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.35/images/create?tag=sha256%3Ab13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691&fromImage=docker.io%2Fnitaigao%2Faido-submissions

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 65, in docker_pull
    pulling = client.api.pull(repository=repository, tag=br.tag, stream=True, decode=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/image.py", line 415, in pull
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 263, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.ImageNotFound: 404 Client Error: Not Found ("pull access denied for nitaigao/aido-submissions, repository does not exist or may require 'docker login': denied: requested access to the resource is denied")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 38, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    raise PullError(msg) from e
duckietown_build_utils.docker_pulling.PullError: Cannot pull repo  docker.io/nitaigao/aido-submissions@sha256:b13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691  tag  None

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 745, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 944, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 42, in docker_pull_retry
    raise PullError(msg) from e
duckietown_build_utils.docker_pulling.PullError: After trying 4 I still could not pull docker.io/nitaigao/aido-submissions@sha256:b13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
7972213798Nicholas Kostelniktemplate-randomaido-hello-sim-validation370abortednonogpu-production-b-spot-0-020:00:22
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 261, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 941, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.35/images/create?tag=sha256%3Ab13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691&fromImage=docker.io%2Fnitaigao%2Faido-submissions

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 65, in docker_pull
    pulling = client.api.pull(repository=repository, tag=br.tag, stream=True, decode=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/image.py", line 415, in pull
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 263, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.ImageNotFound: 404 Client Error: Not Found ("pull access denied for nitaigao/aido-submissions, repository does not exist or may require 'docker login': denied: requested access to the resource is denied")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 38, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    raise PullError(msg) from e
duckietown_build_utils.docker_pulling.PullError: Cannot pull repo  docker.io/nitaigao/aido-submissions@sha256:b13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691  tag  None

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 745, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 944, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 42, in docker_pull_retry
    raise PullError(msg) from e
duckietown_build_utils.docker_pulling.PullError: After trying 4 I still could not pull docker.io/nitaigao/aido-submissions@sha256:b13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
7970016092Matteo Razzanellicollision-check-exercisemooc-collision-check-teststep1successnonogpu-production-b-spot-0-020:00:35
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
false_negative_rate0.3
false_positive_rate0.01


other stats
nsamples200
No reset possible
7969416081Evanthia Fasoulaexercises_braitenbergmooc-BV1sim-4of5successnonogpu-production-b-spot-0-020:01:24
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
distance-from-start_mean1.7690791025684809


other stats
agent_compute-ego0_max0.00987988359787885
agent_compute-ego0_mean0.00987988359787885
agent_compute-ego0_median0.00987988359787885
agent_compute-ego0_min0.00987988359787885
complete-iteration_max0.281692715252147
complete-iteration_mean0.281692715252147
complete-iteration_median0.281692715252147
complete-iteration_min0.281692715252147
deviation-center-line_max0.0
deviation-center-line_mean0.0
deviation-center-line_median0.0
deviation-center-line_min0.0
deviation-heading_max0.0
deviation-heading_mean0.0
deviation-heading_median0.0
deviation-heading_min0.0
distance-from-start_max1.7690791025684809
distance-from-start_median1.7690791025684809
distance-from-start_min1.7690791025684809
driven_any_max1.798189669993195
driven_any_mean1.798189669993195
driven_any_median1.798189669993195
driven_any_min1.798189669993195
driven_lanedir_consec_max0.0
driven_lanedir_consec_mean0.0
driven_lanedir_consec_median0.0
driven_lanedir_consec_min0.0
driven_lanedir_max0.0
driven_lanedir_mean0.0
driven_lanedir_median0.0
driven_lanedir_min0.0
get_duckie_state_max0.11692169413847082
get_duckie_state_mean0.11692169413847082
get_duckie_state_median0.11692169413847082
get_duckie_state_min0.11692169413847082
get_robot_state_max0.003497951170977424
get_robot_state_mean0.003497951170977424
get_robot_state_median0.003497951170977424
get_robot_state_min0.003497951170977424
get_state_dump_max0.020528919556561637
get_state_dump_mean0.020528919556561637
get_state_dump_median0.020528919556561637
get_state_dump_min0.020528919556561637
get_ui_image_max0.04104978056514964
get_ui_image_mean0.04104978056514964
get_ui_image_median0.04104978056514964
get_ui_image_min0.04104978056514964
in-drivable-lane_max4.199999999999993
in-drivable-lane_mean4.199999999999993
in-drivable-lane_median4.199999999999993
in-drivable-lane_min4.199999999999993
per-episodes
details{"d60-ego0": {"driven_any": 1.798189669993195, "get_ui_image": 0.04104978056514964, "step_physics": 0.07439902810489431, "survival_time": 4.199999999999993, "driven_lanedir": 0.0, "get_state_dump": 0.020528919556561637, "get_robot_state": 0.003497951170977424, "sim_render-ego0": 0.00333557689891142, "get_duckie_state": 0.11692169413847082, "in-drivable-lane": 4.199999999999993, "deviation-heading": 0.0, "agent_compute-ego0": 0.00987988359787885, "complete-iteration": 0.281692715252147, "set_robot_commands": 0.002013492584228516, "distance-from-start": 1.7690791025684809, "deviation-center-line": 0.0, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.008140095542458927, "sim_compute_performance-ego0": 0.0018218601451200596}}
set_robot_commands_max0.002013492584228516
set_robot_commands_mean0.002013492584228516
set_robot_commands_median0.002013492584228516
set_robot_commands_min0.002013492584228516
sim_compute_performance-ego0_max0.0018218601451200596
sim_compute_performance-ego0_mean0.0018218601451200596
sim_compute_performance-ego0_median0.0018218601451200596
sim_compute_performance-ego0_min0.0018218601451200596
sim_compute_sim_state_max0.008140095542458927
sim_compute_sim_state_mean0.008140095542458927
sim_compute_sim_state_median0.008140095542458927
sim_compute_sim_state_min0.008140095542458927
sim_render-ego0_max0.00333557689891142
sim_render-ego0_mean0.00333557689891142
sim_render-ego0_median0.00333557689891142
sim_render-ego0_min0.00333557689891142
simulation-passed1
step_physics_max0.07439902810489431
step_physics_mean0.07439902810489431
step_physics_median0.07439902810489431
step_physics_min0.07439902810489431
survival_time_max4.199999999999993
survival_time_mean4.199999999999993
survival_time_median4.199999999999993
survival_time_min4.199999999999993
No reset possible
7967616081Evanthia Fasoulaexercises_braitenbergmooc-BV1sim-4of5successnonogpu-production-b-spot-0-020:02:24
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
distance-from-start_mean5.225518086840181


other stats
agent_compute-ego0_max0.01036892072209772
agent_compute-ego0_mean0.01036892072209772
agent_compute-ego0_median0.01036892072209772
agent_compute-ego0_min0.01036892072209772
complete-iteration_max0.2674343687183452
complete-iteration_mean0.2674343687183452
complete-iteration_median0.2674343687183452
complete-iteration_min0.2674343687183452
deviation-center-line_max0.0
deviation-center-line_mean0.0
deviation-center-line_median0.0
deviation-center-line_min0.0
deviation-heading_max0.0
deviation-heading_mean0.0
deviation-heading_median0.0
deviation-heading_min0.0
distance-from-start_max5.225518086840181
distance-from-start_median5.225518086840181
distance-from-start_min5.225518086840181
driven_any_max5.417676017414257
driven_any_mean5.417676017414257
driven_any_median5.417676017414257
driven_any_min5.417676017414257
driven_lanedir_consec_max0.0
driven_lanedir_consec_mean0.0
driven_lanedir_consec_median0.0
driven_lanedir_consec_min0.0
driven_lanedir_max0.0
driven_lanedir_mean0.0
driven_lanedir_median0.0
driven_lanedir_min0.0
get_duckie_state_max0.10864542677717388
get_duckie_state_mean0.10864542677717388
get_duckie_state_median0.10864542677717388
get_duckie_state_min0.10864542677717388
get_robot_state_max0.003429148557051173
get_robot_state_mean0.003429148557051173
get_robot_state_median0.003429148557051173
get_robot_state_min0.003429148557051173
get_state_dump_max0.020447023634640675
get_state_dump_mean0.020447023634640675
get_state_dump_median0.020447023634640675
get_state_dump_min0.020447023634640675
get_ui_image_max0.041161991515249574
get_ui_image_mean0.041161991515249574
get_ui_image_median0.041161991515249574
get_ui_image_min0.041161991515249574
in-drivable-lane_max10.550000000000017
in-drivable-lane_mean10.550000000000017
in-drivable-lane_median10.550000000000017
in-drivable-lane_min10.550000000000017
per-episodes
details{"d60-ego0": {"driven_any": 5.417676017414257, "get_ui_image": 0.041161991515249574, "step_physics": 0.06949659338537252, "survival_time": 10.550000000000017, "driven_lanedir": 0.0, "get_state_dump": 0.020447023634640675, "get_robot_state": 0.003429148557051173, "sim_render-ego0": 0.0032481877308971475, "get_duckie_state": 0.10864542677717388, "in-drivable-lane": 10.550000000000017, "deviation-heading": 0.0, "agent_compute-ego0": 0.01036892072209772, "complete-iteration": 0.2674343687183452, "set_robot_commands": 0.0020103780728466104, "distance-from-start": 5.225518086840181, "deviation-center-line": 0.0, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.006788859952170894, "sim_compute_performance-ego0": 0.0017414115509896907}}
set_robot_commands_max0.0020103780728466104
set_robot_commands_mean0.0020103780728466104
set_robot_commands_median0.0020103780728466104
set_robot_commands_min0.0020103780728466104
sim_compute_performance-ego0_max0.0017414115509896907
sim_compute_performance-ego0_mean0.0017414115509896907
sim_compute_performance-ego0_median0.0017414115509896907
sim_compute_performance-ego0_min0.0017414115509896907
sim_compute_sim_state_max0.006788859952170894
sim_compute_sim_state_mean0.006788859952170894
sim_compute_sim_state_median0.006788859952170894
sim_compute_sim_state_min0.006788859952170894
sim_render-ego0_max0.0032481877308971475
sim_render-ego0_mean0.0032481877308971475
sim_render-ego0_median0.0032481877308971475
sim_render-ego0_min0.0032481877308971475
simulation-passed1
step_physics_max0.06949659338537252
step_physics_mean0.06949659338537252
step_physics_median0.06949659338537252
step_physics_min0.06949659338537252
survival_time_max10.550000000000017
survival_time_mean10.550000000000017
survival_time_median10.550000000000017
survival_time_min10.550000000000017
No reset possible
7965316094Rohan Kaushikexercises_braitenbergmooc-BV1sim-1of5successnonogpu-production-b-spot-0-020:05:29
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
distance-from-start_mean1.904194532557549


other stats
agent_compute-ego0_max0.010480392963514416
agent_compute-ego0_mean0.010480392963514416
agent_compute-ego0_median0.010480392963514416
agent_compute-ego0_min0.010480392963514416
complete-iteration_max0.2525825095832895
complete-iteration_mean0.2525825095832895
complete-iteration_median0.2525825095832895
complete-iteration_min0.2525825095832895
deviation-center-line_max0.0
deviation-center-line_mean0.0
deviation-center-line_median0.0
deviation-center-line_min0.0
deviation-heading_max0.0
deviation-heading_mean0.0
deviation-heading_median0.0
deviation-heading_min0.0
distance-from-start_max1.904194532557549
distance-from-start_median1.904194532557549
distance-from-start_min1.904194532557549
driven_any_max1.9423198413551872
driven_any_mean1.9423198413551872
driven_any_median1.9423198413551872
driven_any_min1.9423198413551872
driven_lanedir_consec_max0.0
driven_lanedir_consec_mean0.0
driven_lanedir_consec_median0.0
driven_lanedir_consec_min0.0
driven_lanedir_max0.0
driven_lanedir_mean0.0
driven_lanedir_median0.0
driven_lanedir_min0.0
get_duckie_state_max0.09132060326567484
get_duckie_state_mean0.09132060326567484
get_duckie_state_median0.09132060326567484
get_duckie_state_min0.09132060326567484
get_robot_state_max0.00356543118800592
get_robot_state_mean0.00356543118800592
get_robot_state_median0.00356543118800592
get_robot_state_min0.00356543118800592
get_state_dump_max0.01861161813823455
get_state_dump_mean0.01861161813823455
get_state_dump_median0.01861161813823455
get_state_dump_min0.01861161813823455
get_ui_image_max0.039842776202280585
get_ui_image_mean0.039842776202280585
get_ui_image_median0.039842776202280585
get_ui_image_min0.039842776202280585
in-drivable-lane_max21.750000000000178
in-drivable-lane_mean21.750000000000178
in-drivable-lane_median21.750000000000178
in-drivable-lane_min21.750000000000178
per-episodes
details{"d50-ego0": {"driven_any": 1.9423198413551872, "get_ui_image": 0.039842776202280585, "step_physics": 0.07247956138138377, "survival_time": 21.750000000000178, "driven_lanedir": 0.0, "get_state_dump": 0.01861161813823455, "get_robot_state": 0.00356543118800592, "sim_render-ego0": 0.003259224629183428, "get_duckie_state": 0.09132060326567484, "in-drivable-lane": 21.750000000000178, "deviation-heading": 0.0, "agent_compute-ego0": 0.010480392963514416, "complete-iteration": 0.2525825095832895, "set_robot_commands": 0.002014221948221189, "distance-from-start": 1.904194532557549, "deviation-center-line": 0.0, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.00923204859462353, "sim_compute_performance-ego0": 0.0016948197959759913}}
set_robot_commands_max0.002014221948221189
set_robot_commands_mean0.002014221948221189
set_robot_commands_median0.002014221948221189
set_robot_commands_min0.002014221948221189
sim_compute_performance-ego0_max0.0016948197959759913
sim_compute_performance-ego0_mean0.0016948197959759913
sim_compute_performance-ego0_median0.0016948197959759913
sim_compute_performance-ego0_min0.0016948197959759913
sim_compute_sim_state_max0.00923204859462353
sim_compute_sim_state_mean0.00923204859462353
sim_compute_sim_state_median0.00923204859462353
sim_compute_sim_state_min0.00923204859462353
sim_render-ego0_max0.003259224629183428
sim_render-ego0_mean0.003259224629183428
sim_render-ego0_median0.003259224629183428
sim_render-ego0_min0.003259224629183428
simulation-passed1
step_physics_max0.07247956138138377
step_physics_mean0.07247956138138377
step_physics_median0.07247956138138377
step_physics_min0.07247956138138377
survival_time_max21.750000000000178
survival_time_mean21.750000000000178
survival_time_median21.750000000000178
survival_time_min21.750000000000178
No reset possible