Duckietown Challenges Home Challenges Submissions

Evaluator 2116

ID2116
evaluatorreg02-5228613d01c0-1
ownerI don't have one 😀
machinearchimede
processreg02-5228613d01c0-1
versiond-c:6.1.19;d-c-r:6.0.48
first heard
last heard
statusinactive
# evaluating
# success1 36085
# timeout1 36100
# failed
# error1 36095
# aborted1 36088
# host-error
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
361009948Bea Baselines 🐤baseline-duckietownaido5-LFVI_multi-sim-validationLFVIv-simtimeoutnoreg02-5228613d01c0-11:05:31
I can see how the jo [...]
I can see how the job 36100 is timeout because passed 3931 seconds and the timeout is 3600.0.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
360959935Bea Baselines 🐤template-tensorflowaido5-LFV-sim-testingLFVv-simerrornoreg02-5228613d01c0-10:01:05
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "experiment_manager.py", line 196, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 141, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 305, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego" aborted with the following error:

error in ego |Unexpected error:
             |
             || Traceback (most recent call last):
             ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
             ||     call_if_fun_exists(node, "init", context=context_data)
             ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
             ||     f(**kwargs)
             ||   File "solution.py", line 32, in init
             ||     raise Exception(msg)
             || Exception: Could not find gpu device.
             ||
             || The above exception was the direct cause of the following exception:
             ||
             || Traceback (most recent call last):
             ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
             ||     raise Exception(msg) from e
             || Exception: Exception while calling the node's init() function.
             ||
             || | Traceback (most recent call last):
             || |   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
             || |     call_if_fun_exists(node, "init", context=context_data)
             || |   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
             || |     f(**kwargs)
             || |   File "solution.py", line 32, in init
             || |     raise Exception(msg)
             || | Exception: Could not find gpu device.
             || |
             ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 678, in scoring_context
    yield cie
  File "experiment_manager.py", line 655, in go
    wrap(cie)
  File "experiment_manager.py", line 640, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 311, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
360889933Bea Baselines 🐤template-tensorflowaido5-LF-sim-testingLFt-simabortednoreg02-5228613d01c0-10:12:46
KeyboardInterrupt: T [...]
KeyboardInterrupt:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.48-py3.8.egg/duckietown_challenges_runner/runner.py", line 1065, in run_one
    heartbeat()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.48-py3.8.egg/duckietown_challenges_runner/runner.py", line 525, in heartbeat
    raise KeyboardInterrupt(msg)
KeyboardInterrupt: The server told us to abort the job because: The challenge has been updated.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
360859939Bea Baselines 🐤template-tensorflowaido5-LFVI_multi-sim-validationLFVIv-simsuccessnoreg02-5228613d01c0-10:03:18
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.3447367507720487
survival_time_median1.2000000000000004
deviation-center-line_median0.07109987765938118
in-drivable-lane_median0.0


other stats
agent_compute-ego0_max0.06016974647839864
agent_compute-ego0_mean0.06016974647839864
agent_compute-ego0_median0.06016974647839864
agent_compute-ego0_min0.06016974647839864
agent_compute-ego1_max0.05662508805592855
agent_compute-ego1_mean0.05662508805592855
agent_compute-ego1_median0.05662508805592855
agent_compute-ego1_min0.05662508805592855
agent_compute-ego2_max0.06243751446406046
agent_compute-ego2_mean0.06243751446406046
agent_compute-ego2_median0.06243751446406046
agent_compute-ego2_min0.06243751446406046
complete-iteration_max1.3104132811228435
complete-iteration_mean1.3104132811228435
complete-iteration_median1.3104132811228435
complete-iteration_min1.3104132811228435
deviation-center-line_max0.07748043716726215
deviation-center-line_mean0.06400207605410356
deviation-center-line_min0.04342591333566733
deviation-heading_max0.38273968597041863
deviation-heading_mean0.2507722981537057
deviation-heading_median0.3332161416006844
deviation-heading_min0.036361066890014046
driven_any_max0.4584023205286347
driven_any_mean0.3746786451618571
driven_any_median0.3865232510053645
driven_any_min0.27911036395157224
driven_lanedir_consec_max0.4579548009455112
driven_lanedir_consec_mean0.3505233358778
driven_lanedir_consec_min0.24887845591584012
driven_lanedir_max0.4579548009455112
driven_lanedir_mean0.3505233358778
driven_lanedir_median0.3447367507720487
driven_lanedir_min0.24887845591584012
get_state_dump_max0.052513698736826576
get_state_dump_mean0.052513698736826576
get_state_dump_median0.052513698736826576
get_state_dump_min0.052513698736826576
get_ui_image_max0.13278255860010782
get_ui_image_mean0.13278255860010782
get_ui_image_median0.13278255860010782
get_ui_image_min0.13278255860010782
in-drivable-lane_max0.05000000000000005
in-drivable-lane_mean0.01666666666666668
in-drivable-lane_min0.0
per-episodes
details{"udem1-sc0-0-ego0": {"driven_any": 0.27911036395157224, "get_ui_image": 0.13278255860010782, "step_physics": 0.6943288544813792, "survival_time": 1.2000000000000004, "driven_lanedir": 0.24887845591584012, "get_state_dump": 0.052513698736826576, "sim_render-ego0": 0.010009706020355225, "sim_render-ego1": 0.01058278481165568, "sim_render-ego2": 0.015279571215311686, "in-drivable-lane": 0.0, "deviation-heading": 0.38273968597041863, "agent_compute-ego0": 0.06016974647839864, "agent_compute-ego1": 0.05662508805592855, "agent_compute-ego2": 0.06243751446406046, "complete-iteration": 1.3104132811228435, "set_robot_commands": 0.007204532623291016, "deviation-center-line": 0.07109987765938118, "driven_lanedir_consec": 0.24887845591584012, "sim_compute_sim_state": 0.04846528172492981, "sim_compute_performance-ego0": 0.006415635347366333, "sim_compute_performance-ego1": 0.008697787920633951, "sim_compute_performance-ego2": 0.007608513037363688, "sim_compute_robot_state-ego0": 0.017267495393753052, "sim_compute_robot_state-ego1": 0.011144737402598064, "sim_compute_robot_state-ego2": 0.011049191157023111, "sim_compute_robot_state-parked0": 0.01657201846440633, "sim_compute_robot_state-parked1": 0.012129972378412882, "sim_compute_robot_state-parked2": 0.011108219623565674, "sim_compute_robot_state-parked3": 0.011206885178883873, "sim_compute_robot_state-parked4": 0.020489672819773357, "sim_compute_robot_state-parked5": 0.010911295811335243}, "udem1-sc0-0-ego1": {"driven_any": 0.4584023205286347, "get_ui_image": 0.13278255860010782, "step_physics": 0.6943288544813792, "survival_time": 1.2000000000000004, "driven_lanedir": 0.4579548009455112, "get_state_dump": 0.052513698736826576, "sim_render-ego0": 0.010009706020355225, "sim_render-ego1": 0.01058278481165568, "sim_render-ego2": 0.015279571215311686, "in-drivable-lane": 0.0, "deviation-heading": 0.036361066890014046, "agent_compute-ego0": 0.06016974647839864, "agent_compute-ego1": 0.05662508805592855, "agent_compute-ego2": 0.06243751446406046, "complete-iteration": 1.3104132811228435, "set_robot_commands": 0.007204532623291016, "deviation-center-line": 0.04342591333566733, "driven_lanedir_consec": 0.4579548009455112, "sim_compute_sim_state": 0.04846528172492981, "sim_compute_performance-ego0": 0.006415635347366333, "sim_compute_performance-ego1": 0.008697787920633951, "sim_compute_performance-ego2": 0.007608513037363688, "sim_compute_robot_state-ego0": 0.017267495393753052, "sim_compute_robot_state-ego1": 0.011144737402598064, "sim_compute_robot_state-ego2": 0.011049191157023111, "sim_compute_robot_state-parked0": 0.01657201846440633, "sim_compute_robot_state-parked1": 0.012129972378412882, "sim_compute_robot_state-parked2": 0.011108219623565674, "sim_compute_robot_state-parked3": 0.011206885178883873, "sim_compute_robot_state-parked4": 0.020489672819773357, "sim_compute_robot_state-parked5": 0.010911295811335243}, "udem1-sc0-0-ego2": {"driven_any": 0.3865232510053645, "get_ui_image": 0.13278255860010782, "step_physics": 0.6943288544813792, "survival_time": 1.2000000000000004, "driven_lanedir": 0.3447367507720487, "get_state_dump": 0.052513698736826576, "sim_render-ego0": 0.010009706020355225, "sim_render-ego1": 0.01058278481165568, "sim_render-ego2": 0.015279571215311686, "in-drivable-lane": 0.05000000000000005, "deviation-heading": 0.3332161416006844, "agent_compute-ego0": 0.06016974647839864, "agent_compute-ego1": 0.05662508805592855, "agent_compute-ego2": 0.06243751446406046, "complete-iteration": 1.3104132811228435, "set_robot_commands": 0.007204532623291016, "deviation-center-line": 0.07748043716726215, "driven_lanedir_consec": 0.3447367507720487, "sim_compute_sim_state": 0.04846528172492981, "sim_compute_performance-ego0": 0.006415635347366333, "sim_compute_performance-ego1": 0.008697787920633951, "sim_compute_performance-ego2": 0.007608513037363688, "sim_compute_robot_state-ego0": 0.017267495393753052, "sim_compute_robot_state-ego1": 0.011144737402598064, "sim_compute_robot_state-ego2": 0.011049191157023111, "sim_compute_robot_state-parked0": 0.01657201846440633, "sim_compute_robot_state-parked1": 0.012129972378412882, "sim_compute_robot_state-parked2": 0.011108219623565674, "sim_compute_robot_state-parked3": 0.011206885178883873, "sim_compute_robot_state-parked4": 0.020489672819773357, "sim_compute_robot_state-parked5": 0.010911295811335243}}
set_robot_commands_max0.007204532623291016
set_robot_commands_mean0.007204532623291016
set_robot_commands_median0.007204532623291016
set_robot_commands_min0.007204532623291016
sim_compute_performance-ego0_max0.006415635347366333
sim_compute_performance-ego0_mean0.006415635347366333
sim_compute_performance-ego0_median0.006415635347366333
sim_compute_performance-ego0_min0.006415635347366333
sim_compute_performance-ego1_max0.008697787920633951
sim_compute_performance-ego1_mean0.008697787920633951
sim_compute_performance-ego1_median0.008697787920633951
sim_compute_performance-ego1_min0.008697787920633951
sim_compute_performance-ego2_max0.007608513037363688
sim_compute_performance-ego2_mean0.007608513037363688
sim_compute_performance-ego2_median0.007608513037363688
sim_compute_performance-ego2_min0.007608513037363688
sim_compute_robot_state-ego0_max0.017267495393753052
sim_compute_robot_state-ego0_mean0.017267495393753052
sim_compute_robot_state-ego0_median0.017267495393753052
sim_compute_robot_state-ego0_min0.017267495393753052
sim_compute_robot_state-ego1_max0.011144737402598064
sim_compute_robot_state-ego1_mean0.011144737402598064
sim_compute_robot_state-ego1_median0.011144737402598064
sim_compute_robot_state-ego1_min0.011144737402598064
sim_compute_robot_state-ego2_max0.011049191157023111
sim_compute_robot_state-ego2_mean0.011049191157023111
sim_compute_robot_state-ego2_median0.011049191157023111
sim_compute_robot_state-ego2_min0.011049191157023111
sim_compute_robot_state-parked0_max0.01657201846440633
sim_compute_robot_state-parked0_mean0.01657201846440633
sim_compute_robot_state-parked0_median0.01657201846440633
sim_compute_robot_state-parked0_min0.01657201846440633
sim_compute_robot_state-parked1_max0.012129972378412882
sim_compute_robot_state-parked1_mean0.012129972378412882
sim_compute_robot_state-parked1_median0.012129972378412882
sim_compute_robot_state-parked1_min0.012129972378412882
sim_compute_robot_state-parked2_max0.011108219623565674
sim_compute_robot_state-parked2_mean0.011108219623565674
sim_compute_robot_state-parked2_median0.011108219623565674
sim_compute_robot_state-parked2_min0.011108219623565674
sim_compute_robot_state-parked3_max0.011206885178883873
sim_compute_robot_state-parked3_mean0.011206885178883873
sim_compute_robot_state-parked3_median0.011206885178883873
sim_compute_robot_state-parked3_min0.011206885178883873
sim_compute_robot_state-parked4_max0.020489672819773357
sim_compute_robot_state-parked4_mean0.020489672819773357
sim_compute_robot_state-parked4_median0.020489672819773357
sim_compute_robot_state-parked4_min0.020489672819773357
sim_compute_robot_state-parked5_max0.010911295811335243
sim_compute_robot_state-parked5_mean0.010911295811335243
sim_compute_robot_state-parked5_median0.010911295811335243
sim_compute_robot_state-parked5_min0.010911295811335243
sim_compute_sim_state_max0.04846528172492981
sim_compute_sim_state_mean0.04846528172492981
sim_compute_sim_state_median0.04846528172492981
sim_compute_sim_state_min0.04846528172492981
sim_render-ego0_max0.010009706020355225
sim_render-ego0_mean0.010009706020355225
sim_render-ego0_median0.010009706020355225
sim_render-ego0_min0.010009706020355225
sim_render-ego1_max0.01058278481165568
sim_render-ego1_mean0.01058278481165568
sim_render-ego1_median0.01058278481165568
sim_render-ego1_min0.01058278481165568
sim_render-ego2_max0.015279571215311686
sim_render-ego2_mean0.015279571215311686
sim_render-ego2_median0.015279571215311686
sim_render-ego2_min0.015279571215311686
simulation-passed1
step_physics_max0.6943288544813792
step_physics_mean0.6943288544813792
step_physics_median0.6943288544813792
step_physics_min0.6943288544813792
survival_time_max1.2000000000000004
survival_time_mean1.2000000000000004
survival_time_min1.2000000000000004
No reset possible