Duckietown Challenges Home Challenges Submissions

Evaluator 203

ID203
evaluatorp-16753
ownerAndrea Censi 🇨🇭
machinedorothy
processp-16753
versionDC:3.1.47;DCR:3.2.10;DTS:3.0.29
first heard
last heard
statusinactive
# evaluating
# success5 2724
# timeout4 2706
# failed2 2720
# error2 2756
# aborted1 2690
# host-error
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
2763304Anton Mashikhin 🇷🇺AI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step1-simulationerrornop-167530:12:53
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/data/work/vmware-michela-shared/DT/dt-env/dt-env-developer/challenges/duckietown-challenges-runner/src/duckietown_challenges_runner/runner.py", line 301, in go_
    cr = run_single(wd, aws_config, steps2artefacts, challenge_parameters_, solution_container, challenge_name,
  File "/data/work/vmware-michela-shared/DT/dt-env/dt-env-developer/challenges/duckietown-challenges-runner/src/duckietown_challenges_runner/runner.py", line 388, in run_single
  File "/data/work/vmware-michela-shared/DT/dt-env/dt-env-developer/challenges/duckietown-challenges-runner/src/duckietown_challenges_runner/runner.py", line 631, in write_logs
    logs = logs_for_container(client, container_id)
  File "/data/work/vmware-michela-shared/DT/dt-env/dt-env-developer/challenges/duckietown-challenges-runner/src/duckietown_challenges_runner/runner.py", line 781, in logs_for_container
    # for c in container.logs(stdout=True, stderr=True, stream=True, timestamps=True):
  File "/Users/andrea/env_mcdp/deploy/lib/python2.7/site-packages/docker/types/daemon.py", line 30, in __next__
    return next(self._stream)
  File "/Users/andrea/env_mcdp/deploy/lib/python2.7/site-packages/docker/api/client.py", line 348, in _multiplexed_response_stream_helper
    header = response.raw.read(STREAM_HEADER_SIZE_BYTES)
  File "/Users/andrea/env_mcdp/deploy/lib/python2.7/site-packages/urllib3/response.py", line 384, in read
    data = self._fp.read(amt)
  File "/Applications/Canopy.app/appdata/canopy-1.7.4.3348.macosx-x86_64/Canopy.app/Contents/lib/python2.7/httplib.py", line 588, in read
    return self._read_chunked(amt)
  File "/Applications/Canopy.app/appdata/canopy-1.7.4.3348.macosx-x86_64/Canopy.app/Contents/lib/python2.7/httplib.py", line 630, in _read_chunked
    line = self.fp.readline(_MAXLINE + 1)
  File "/Applications/Canopy.app/appdata/canopy-1.7.4.3348.macosx-x86_64/Canopy.app/Contents/lib/python2.7/socket.py", line 480, in readline
    data = self._sock.recv(self._rbufsize)
KeyboardInterrupt
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
2756317Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationerrornop-167530:10:27
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/data/work/vmware-michela-shared/DT/dt-env/dt-env-developer/challenges/duckietown-challenges-runner/src/duckietown_challenges_runner/runner.py", line 507, in run_one
  File "/data/work/vmware-michela-shared/DT/dt-env/dt-env-developer/challenges/duckietown-challenges/src/duckietown_challenges/challenge_results.py", line 63, in read_challenge_results
    raise Exception(msg)
Exception: File u'/tmp/duckietown/DT18/evaluator/executions/aido1_LF1_r3-v3/submission317/step1-simulation-p-16753-job2756/challenge-results/challenge_results.yaml' does not exist.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
2754318Manfred DiazTensorflow templateaido1_LF1_r3-v3step1-simulationfailednop-167530:00:35
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 586, in wrap_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Uncaught exception in solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 581, in wrap_solution
    solution.run(cis)
  File "solution.py", line 61, in run
    solve(params, cis)  # let's try to solve the challenge, exciting ah?
  File "solution.py", line 35, in solve
    action = model.predict(observation)
  File "/workspace/model.py", line 64, in predict
    self._observation: [state],
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 887, in run
    run_metadata_ptr)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1086, in _run
    str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (1, 120, 160, 3) for Tensor u'state:0', which has shape '(1, 480, 640, 3)'


Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
2747307Andrea Censi 🇨🇭Random executionaido1_LF1_r3-v3step4-vizsuccessnop-167530:04:23
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
No reset possible
2746319Julian ZillyBaseline solution using imitation learning from logsaido1_LF1_r3-v3step4-vizsuccessnop-167530:01:59
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
No reset possible
2742319Julian ZillyBaseline solution using imitation learning from logsaido1_LF1_r3-v3step3-videossuccessnop-167530:01:44
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
No reset possible
2741319Julian ZillyBaseline solution using imitation learning from logsaido1_LF1_r3-v3step2-scoringsuccessnop-167530:00:29
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median16.666666666666654


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -0.5091864165328444, "good_angle": 57.43752411062863, "survival_time": 16.666666666666654, "traveled_tiles": 3, "valid_direction": 6.866666666666643}, "ep001": {"nsteps": 500, "reward": -0.5651936712861061, "good_angle": 2.422744287958935, "survival_time": 16.666666666666654, "traveled_tiles": 3, "valid_direction": 0.3000000000000149}, "ep002": {"nsteps": 500, "reward": -0.5114709859639406, "good_angle": 0.09252956807780274, "survival_time": 16.666666666666654, "traveled_tiles": 4, "valid_direction": 0}, "ep003": {"nsteps": 500, "reward": -0.776356368791312, "good_angle": 57.65536251214368, "survival_time": 16.666666666666654, "traveled_tiles": 4, "valid_direction": 6.899999999999979}, "ep004": {"nsteps": 129, "reward": -10.401626305986744, "good_angle": 0.3925861772498824, "survival_time": 4.2999999999999945, "traveled_tiles": 2, "valid_direction": 1.2000000000000024}}
good_angle_max57.65536251214368
good_angle_mean23.60014933121179
good_angle_median2.422744287958935
good_angle_min0.09252956807780274
reward_max-0.5091864165328444
reward_mean-2.5527667497121893
reward_median-0.5651936712861061
reward_min-10.401626305986744
survival_time_max16.666666666666654
survival_time_mean14.193333333333324
survival_time_min4.2999999999999945
traveled_tiles_max4
traveled_tiles_mean3.2
traveled_tiles_median3
traveled_tiles_min2
valid_direction_max6.899999999999979
valid_direction_mean3.0533333333333283
valid_direction_median1.2000000000000024
valid_direction_min0
No reset possible
2724319Julian ZillyBaseline solution using imitation learning from logsaido1_LF1_r3-v3step1-simulationsuccessnop-167530:04:55
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
No reset possible
2720165Andrea Censi 🇨🇭Random executionaido1_LF1_r3-v3step1-simulationfailednop-167530:00:32
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 580, in wrap_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Uncaught exception in solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 575, in wrap_solution
    solution.run(cis)
  File "solution.py", line 64, in run
    solve(gym_environment, cis)  # let's try to solve the challenge, exciting ah?
  File "solution.py", line 39, in solve
    observation, reward, done, info = env.step(action)
  File "/usr/local/lib/python2.7/dist-packages/gym/wrappers/time_limit.py", line 31, in step
    observation, reward, done, info = self.env.step(action)
  File "/notebooks/src/gym-duckietown-agent/gym_duckietown_agent/envs/simplesimagent_env.py", line 107, in step
    obs, rew, done, misc = self.sim.step(action, with_observation=True)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 52, in step
    return self._failsafe_observe(msg)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 74, in _failsafe_observe
    raise Exception("Can't connect to the gym-duckietown-server")
Exception: Can't connect to the gym-duckietown-server


Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
2715268Andrea Censi 🇨🇭Random executionaido1_LFV_r1-v3step4-viztimeoutnop-167530:11:02
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
2712305Andrea Censi 🇨🇭Random executionaido1_LF1_r3-v3step4-viztimeoutnop-167530:11:06
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
2709305Andrea Censi 🇨🇭Random executionaido1_LF1_r3-v3step4-viztimeoutnop-167530:11:11
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
2706306Andrea Censi 🇨🇭Random executionaido1_LF1_r3-v3step4-viztimeoutnop-167530:11:22
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
2690166Andrea Censi 🇨🇭Random executionaido1_LFV_r1-v3step1-simulationabortednop-167530:02:56
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
No reset possible