Duckietown Challenges Home Challenges Submissions

Evaluator 5317

ID5317
evaluatorgpu-production-spot-0-01
ownerI don't have one 😀
machinegpu-production-spot-0_a65ee16a73b6
processgpu-production-spot-0-01_a65ee16a73b6
version6.2.7
first heard
last heard
statusinactive
# evaluating
# success4 79205
# timeout
# failed
# error
# aborted1 79194
# host-error
arm0
x86_641
Mac0
gpu available1
Number of processors64
Processor frequency (MHz)0.0 GHz
Free % of processors99%
RAM total (MB)249.0 GB
RAM free (MB)240.7 GB
Disk (MB)969.3 GB
Disk available (MB)870.0 GB
Docker Hub
P11
P2
Cloud simulations1
PI Camera0
# Duckiebots0
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
7949716059Evanthia Fasoulaexercises_braitenbergmooc-BV1sim-1of5successnogpu-production-spot-0-010:02:48
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
distance-from-start_mean1.0665998426753398


other stats
agent_compute-ego0_max0.01223162015279134
agent_compute-ego0_mean0.01223162015279134
agent_compute-ego0_median0.01223162015279134
agent_compute-ego0_min0.01223162015279134
complete-iteration_max0.26258102655410764
complete-iteration_mean0.26258102655410764
complete-iteration_median0.26258102655410764
complete-iteration_min0.26258102655410764
deviation-center-line_max0.0
deviation-center-line_mean0.0
deviation-center-line_median0.0
deviation-center-line_min0.0
deviation-heading_max0.0
deviation-heading_mean0.0
deviation-heading_median0.0
deviation-heading_min0.0
distance-from-start_max1.0665998426753398
distance-from-start_median1.0665998426753398
distance-from-start_min1.0665998426753398
driven_any_max1.0711697451441575
driven_any_mean1.0711697451441575
driven_any_median1.0711697451441575
driven_any_min1.0711697451441575
driven_lanedir_consec_max0.0
driven_lanedir_consec_mean0.0
driven_lanedir_consec_median0.0
driven_lanedir_consec_min0.0
driven_lanedir_max0.0
driven_lanedir_mean0.0
driven_lanedir_median0.0
driven_lanedir_min0.0
get_duckie_state_max0.1196298360824585
get_duckie_state_mean0.1196298360824585
get_duckie_state_median0.1196298360824585
get_duckie_state_min0.1196298360824585
get_robot_state_max0.0036477923393249513
get_robot_state_mean0.0036477923393249513
get_robot_state_median0.0036477923393249513
get_robot_state_min0.0036477923393249513
get_state_dump_max0.02394949992497762
get_state_dump_mean0.02394949992497762
get_state_dump_median0.02394949992497762
get_state_dump_min0.02394949992497762
get_ui_image_max0.01592049201329549
get_ui_image_mean0.01592049201329549
get_ui_image_median0.01592049201329549
get_ui_image_min0.01592049201329549
in-drivable-lane_max2.9499999999999975
in-drivable-lane_mean2.9499999999999975
in-drivable-lane_median2.9499999999999975
in-drivable-lane_min2.9499999999999975
per-episodes
details{"d60-ego0": {"driven_any": 1.0711697451441575, "get_ui_image": 0.01592049201329549, "step_physics": 0.06978729565938314, "survival_time": 2.9499999999999975, "driven_lanedir": 0.0, "get_state_dump": 0.02394949992497762, "get_robot_state": 0.0036477923393249513, "sim_render-ego0": 0.00363459587097168, "get_duckie_state": 0.1196298360824585, "in-drivable-lane": 2.9499999999999975, "deviation-heading": 0.0, "agent_compute-ego0": 0.01223162015279134, "complete-iteration": 0.26258102655410764, "set_robot_commands": 0.0023357470830281577, "distance-from-start": 1.0665998426753398, "deviation-center-line": 0.0, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.009425115585327147, "sim_compute_performance-ego0": 0.0019216696421305336}}
set_robot_commands_max0.0023357470830281577
set_robot_commands_mean0.0023357470830281577
set_robot_commands_median0.0023357470830281577
set_robot_commands_min0.0023357470830281577
sim_compute_performance-ego0_max0.0019216696421305336
sim_compute_performance-ego0_mean0.0019216696421305336
sim_compute_performance-ego0_median0.0019216696421305336
sim_compute_performance-ego0_min0.0019216696421305336
sim_compute_sim_state_max0.009425115585327147
sim_compute_sim_state_mean0.009425115585327147
sim_compute_sim_state_median0.009425115585327147
sim_compute_sim_state_min0.009425115585327147
sim_render-ego0_max0.00363459587097168
sim_render-ego0_mean0.00363459587097168
sim_render-ego0_median0.00363459587097168
sim_render-ego0_min0.00363459587097168
simulation-passed1
step_physics_max0.06978729565938314
step_physics_mean0.06978729565938314
step_physics_median0.06978729565938314
step_physics_min0.06978729565938314
survival_time_max2.9499999999999975
survival_time_mean2.9499999999999975
survival_time_median2.9499999999999975
survival_time_min2.9499999999999975
No reset possible
7947916057Evanthia Fasoulaexercises_braitenbergmooc-BV1sim-1of5successnogpu-production-spot-0-010:02:49
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
distance-from-start_mean1.1025954992125644


other stats
agent_compute-ego0_max0.012304714170552917
agent_compute-ego0_mean0.012304714170552917
agent_compute-ego0_median0.012304714170552917
agent_compute-ego0_min0.012304714170552917
complete-iteration_max0.2539897975275072
complete-iteration_mean0.2539897975275072
complete-iteration_median0.2539897975275072
complete-iteration_min0.2539897975275072
deviation-center-line_max0.0
deviation-center-line_mean0.0
deviation-center-line_median0.0
deviation-center-line_min0.0
deviation-heading_max0.0
deviation-heading_mean0.0
deviation-heading_median0.0
deviation-heading_min0.0
distance-from-start_max1.1025954992125644
distance-from-start_median1.1025954992125644
distance-from-start_min1.1025954992125644
driven_any_max1.102609084384248
driven_any_mean1.102609084384248
driven_any_median1.102609084384248
driven_any_min1.102609084384248
driven_lanedir_consec_max0.0
driven_lanedir_consec_mean0.0
driven_lanedir_consec_median0.0
driven_lanedir_consec_min0.0
driven_lanedir_max0.0
driven_lanedir_mean0.0
driven_lanedir_median0.0
driven_lanedir_min0.0
get_duckie_state_max0.1164319191948842
get_duckie_state_mean0.1164319191948842
get_duckie_state_median0.1164319191948842
get_duckie_state_min0.1164319191948842
get_robot_state_max0.0035730580152091335
get_robot_state_mean0.0035730580152091335
get_robot_state_median0.0035730580152091335
get_robot_state_min0.0035730580152091335
get_state_dump_max0.023203154741707498
get_state_dump_mean0.023203154741707498
get_state_dump_median0.023203154741707498
get_state_dump_min0.023203154741707498
get_ui_image_max0.01581123723822125
get_ui_image_mean0.01581123723822125
get_ui_image_median0.01581123723822125
get_ui_image_min0.01581123723822125
in-drivable-lane_max2.8999999999999977
in-drivable-lane_mean2.8999999999999977
in-drivable-lane_median2.8999999999999977
in-drivable-lane_min2.8999999999999977
per-episodes
details{"d60-ego0": {"driven_any": 1.102609084384248, "get_ui_image": 0.01581123723822125, "step_physics": 0.06603109634528725, "survival_time": 2.8999999999999977, "driven_lanedir": 0.0, "get_state_dump": 0.023203154741707498, "get_robot_state": 0.0035730580152091335, "sim_render-ego0": 0.003557484028703075, "get_duckie_state": 0.1164319191948842, "in-drivable-lane": 2.8999999999999977, "deviation-heading": 0.0, "agent_compute-ego0": 0.012304714170552917, "complete-iteration": 0.2539897975275072, "set_robot_commands": 0.002260689008033882, "distance-from-start": 1.1025954992125644, "deviation-center-line": 0.0, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.008858523126375877, "sim_compute_performance-ego0": 0.001874758025347176}}
set_robot_commands_max0.002260689008033882
set_robot_commands_mean0.002260689008033882
set_robot_commands_median0.002260689008033882
set_robot_commands_min0.002260689008033882
sim_compute_performance-ego0_max0.001874758025347176
sim_compute_performance-ego0_mean0.001874758025347176
sim_compute_performance-ego0_median0.001874758025347176
sim_compute_performance-ego0_min0.001874758025347176
sim_compute_sim_state_max0.008858523126375877
sim_compute_sim_state_mean0.008858523126375877
sim_compute_sim_state_median0.008858523126375877
sim_compute_sim_state_min0.008858523126375877
sim_render-ego0_max0.003557484028703075
sim_render-ego0_mean0.003557484028703075
sim_render-ego0_median0.003557484028703075
sim_render-ego0_min0.003557484028703075
simulation-passed1
step_physics_max0.06603109634528725
step_physics_mean0.06603109634528725
step_physics_median0.06603109634528725
step_physics_min0.06603109634528725
survival_time_max2.8999999999999977
survival_time_mean2.8999999999999977
survival_time_median2.8999999999999977
survival_time_min2.8999999999999977
No reset possible
7925415969Rayed Wahedexercises_braitenbergmooc-BV1sim-2of5successnogpu-production-spot-0-010:03:22
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
distance-from-start_mean4.61022654375225


other stats
agent_compute-ego0_max0.01098575526720857
agent_compute-ego0_mean0.01098575526720857
agent_compute-ego0_median0.01098575526720857
agent_compute-ego0_min0.01098575526720857
complete-iteration_max0.20773441889514663
complete-iteration_mean0.20773441889514663
complete-iteration_median0.20773441889514663
complete-iteration_min0.20773441889514663
deviation-center-line_max0.0
deviation-center-line_mean0.0
deviation-center-line_median0.0
deviation-center-line_min0.0
deviation-heading_max0.0
deviation-heading_mean0.0
deviation-heading_median0.0
deviation-heading_min0.0
distance-from-start_max4.61022654375225
distance-from-start_median4.61022654375225
distance-from-start_min4.61022654375225
driven_any_max5.594961380533402
driven_any_mean5.594961380533402
driven_any_median5.594961380533402
driven_any_min5.594961380533402
driven_lanedir_consec_max0.0
driven_lanedir_consec_mean0.0
driven_lanedir_consec_median0.0
driven_lanedir_consec_min0.0
driven_lanedir_max0.0
driven_lanedir_mean0.0
driven_lanedir_median0.0
driven_lanedir_min0.0
get_duckie_state_max0.07866216424393327
get_duckie_state_mean0.07866216424393327
get_duckie_state_median0.07866216424393327
get_duckie_state_min0.07866216424393327
get_robot_state_max0.003644248230816567
get_robot_state_mean0.003644248230816567
get_robot_state_median0.003644248230816567
get_robot_state_min0.003644248230816567
get_state_dump_max0.01712320144862345
get_state_dump_mean0.01712320144862345
get_state_dump_median0.01712320144862345
get_state_dump_min0.01712320144862345
get_ui_image_max0.014248024274225104
get_ui_image_mean0.014248024274225104
get_ui_image_median0.014248024274225104
get_ui_image_min0.014248024274225104
in-drivable-lane_max18.200000000000124
in-drivable-lane_mean18.200000000000124
in-drivable-lane_median18.200000000000124
in-drivable-lane_min18.200000000000124
per-episodes
details{"d40-ego0": {"driven_any": 5.594961380533402, "get_ui_image": 0.014248024274225104, "step_physics": 0.06735567981249665, "survival_time": 18.200000000000124, "driven_lanedir": 0.0, "get_state_dump": 0.01712320144862345, "get_robot_state": 0.003644248230816567, "sim_render-ego0": 0.0035264420182737587, "get_duckie_state": 0.07866216424393327, "in-drivable-lane": 18.200000000000124, "deviation-heading": 0.0, "agent_compute-ego0": 0.01098575526720857, "complete-iteration": 0.20773441889514663, "set_robot_commands": 0.002220889313580239, "distance-from-start": 4.61022654375225, "deviation-center-line": 0.0, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.008026059686321101, "sim_compute_performance-ego0": 0.0018568829314349448}}
set_robot_commands_max0.002220889313580239
set_robot_commands_mean0.002220889313580239
set_robot_commands_median0.002220889313580239
set_robot_commands_min0.002220889313580239
sim_compute_performance-ego0_max0.0018568829314349448
sim_compute_performance-ego0_mean0.0018568829314349448
sim_compute_performance-ego0_median0.0018568829314349448
sim_compute_performance-ego0_min0.0018568829314349448
sim_compute_sim_state_max0.008026059686321101
sim_compute_sim_state_mean0.008026059686321101
sim_compute_sim_state_median0.008026059686321101
sim_compute_sim_state_min0.008026059686321101
sim_render-ego0_max0.0035264420182737587
sim_render-ego0_mean0.0035264420182737587
sim_render-ego0_median0.0035264420182737587
sim_render-ego0_min0.0035264420182737587
simulation-passed1
step_physics_max0.06735567981249665
step_physics_mean0.06735567981249665
step_physics_median0.06735567981249665
step_physics_min0.06735567981249665
survival_time_max18.200000000000124
survival_time_mean18.200000000000124
survival_time_median18.200000000000124
survival_time_min18.200000000000124
No reset possible
7920515953Shubhra Aichexercises_braitenbergmooc-BV1sim-0of5successnogpu-production-spot-0-010:11:29
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
distance-from-start_mean3.8525944183556735


other stats
agent_compute-ego0_max0.010273125646116337
agent_compute-ego0_mean0.010273125646116337
agent_compute-ego0_median0.010273125646116337
agent_compute-ego0_min0.010273125646116337
complete-iteration_max0.20515181956739847
complete-iteration_mean0.20515181956739847
complete-iteration_median0.20515181956739847
complete-iteration_min0.20515181956739847
deviation-center-line_max0.0
deviation-center-line_mean0.0
deviation-center-line_median0.0
deviation-center-line_min0.0
deviation-heading_max0.0
deviation-heading_mean0.0
deviation-heading_median0.0
deviation-heading_min0.0
distance-from-start_max3.8525944183556735
distance-from-start_median3.8525944183556735
distance-from-start_min3.8525944183556735
driven_any_max4.192923066119024
driven_any_mean4.192923066119024
driven_any_median4.192923066119024
driven_any_min4.192923066119024
driven_lanedir_consec_max0.0
driven_lanedir_consec_mean0.0
driven_lanedir_consec_median0.0
driven_lanedir_consec_min0.0
driven_lanedir_max0.0
driven_lanedir_mean0.0
driven_lanedir_median0.0
driven_lanedir_min0.0
get_duckie_state_max0.08488006337695475
get_duckie_state_mean0.08488006337695475
get_duckie_state_median0.08488006337695475
get_duckie_state_min0.08488006337695475
get_robot_state_max0.0034405865538229452
get_robot_state_mean0.0034405865538229452
get_robot_state_median0.0034405865538229452
get_robot_state_min0.0034405865538229452
get_state_dump_max0.01765254355787139
get_state_dump_mean0.01765254355787139
get_state_dump_median0.01765254355787139
get_state_dump_min0.01765254355787139
get_ui_image_max0.013904340459742611
get_ui_image_mean0.013904340459742611
get_ui_image_median0.013904340459742611
get_ui_image_min0.013904340459742611
in-drivable-lane_max59.99999999999873
in-drivable-lane_mean59.99999999999873
in-drivable-lane_median59.99999999999873
in-drivable-lane_min59.99999999999873
per-episodes
details{"d45-ego0": {"driven_any": 4.192923066119024, "get_ui_image": 0.013904340459742611, "step_physics": 0.05820114090480376, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "get_state_dump": 0.01765254355787139, "get_robot_state": 0.0034405865538229452, "sim_render-ego0": 0.0033636319448707703, "get_duckie_state": 0.08488006337695475, "in-drivable-lane": 59.99999999999873, "deviation-heading": 0.0, "agent_compute-ego0": 0.010273125646116337, "complete-iteration": 0.20515181956739847, "set_robot_commands": 0.00209127020379288, "distance-from-start": 3.8525944183556735, "deviation-center-line": 0.0, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.009483913696378, "sim_compute_performance-ego0": 0.0017888978756436895}}
set_robot_commands_max0.00209127020379288
set_robot_commands_mean0.00209127020379288
set_robot_commands_median0.00209127020379288
set_robot_commands_min0.00209127020379288
sim_compute_performance-ego0_max0.0017888978756436895
sim_compute_performance-ego0_mean0.0017888978756436895
sim_compute_performance-ego0_median0.0017888978756436895
sim_compute_performance-ego0_min0.0017888978756436895
sim_compute_sim_state_max0.009483913696378
sim_compute_sim_state_mean0.009483913696378
sim_compute_sim_state_median0.009483913696378
sim_compute_sim_state_min0.009483913696378
sim_render-ego0_max0.0033636319448707703
sim_render-ego0_mean0.0033636319448707703
sim_render-ego0_median0.0033636319448707703
sim_render-ego0_min0.0033636319448707703
simulation-passed1
step_physics_max0.05820114090480376
step_physics_mean0.05820114090480376
step_physics_median0.05820114090480376
step_physics_min0.05820114090480376
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_median59.99999999999873
survival_time_min59.99999999999873
No reset possible
7919413798Nicholas Kostelniktemplate-randomaido-hello-sim-validation370abortednogpu-production-spot-0-010:00:22
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 261, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 941, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.35/images/create?tag=sha256%3Ab13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691&fromImage=docker.io%2Fnitaigao%2Faido-submissions

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 65, in docker_pull
    pulling = client.api.pull(repository=repository, tag=br.tag, stream=True, decode=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/image.py", line 415, in pull
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 263, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.ImageNotFound: 404 Client Error: Not Found ("pull access denied for nitaigao/aido-submissions, repository does not exist or may require 'docker login': denied: requested access to the resource is denied")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 38, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    raise PullError(msg) from e
duckietown_build_utils.docker_pulling.PullError: Cannot pull repo  docker.io/nitaigao/aido-submissions@sha256:b13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691  tag  None

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 745, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 944, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 42, in docker_pull_retry
    raise PullError(msg) from e
duckietown_build_utils.docker_pulling.PullError: After trying 4 I still could not pull docker.io/nitaigao/aido-submissions@sha256:b13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible