Duckietown Challenges Home Challenges Submissions

Evaluator 221

ID221
evaluatorrudolf-3
ownerAndrea Censi 🇨🇭
machineidsc-rudolf
processrudolf-3
versionDC:3.1.50;DCR:3.2.10;DTS:3.0.29
first heard
last heard
statusinactive
# evaluating
# success5 2842
# timeout
# failed4 2817
# error5 2813
# aborted
# host-error
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
2856336Andrea Censi 🇨🇭Tensorflow templateaido1_LFV_r1-v3step1-simulationfailednorudolf-30:00:44
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 586, in wrap_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Uncaught exception in solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 581, in wrap_solution
    solution.run(cis)
  File "solution.py", line 86, in run
    solve(params, cis)  # let's try to solve the challenge,
  File "solution.py", line 52, in solve
    observation, reward, done, info = env.step(action)
  File "/usr/local/lib/python2.7/dist-packages/gym/wrappers/time_limit.py", line 31, in step
    observation, reward, done, info = self.env.step(action)
  File "/notebooks/src/gym-duckietown-agent/gym_duckietown_agent/envs/simplesimagent_env.py", line 109, in step
    obs, rew, done, misc = self.sim.step(action, with_observation=True)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 55, in step
    return self._failsafe_observe(msg)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 86, in _failsafe_observe
    raise Exception(msg)
Exception: Giving up to connect to the gym duckietown server at host: evaluator
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
2852333Andrea Censi 🇨🇭Tensorflow templateaido1_LF1_r3-v3step1-simulationfailednorudolf-30:00:47
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 586, in wrap_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Uncaught exception in solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 581, in wrap_solution
    solution.run(cis)
  File "solution.py", line 86, in run
    solve(params, cis)  # let's try to solve the challenge,
  File "solution.py", line 52, in solve
    observation, reward, done, info = env.step(action)
  File "/usr/local/lib/python2.7/dist-packages/gym/wrappers/time_limit.py", line 31, in step
    observation, reward, done, info = self.env.step(action)
  File "/notebooks/src/gym-duckietown-agent/gym_duckietown_agent/envs/simplesimagent_env.py", line 109, in step
    obs, rew, done, misc = self.sim.step(action, with_observation=True)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 55, in step
    return self._failsafe_observe(msg)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 86, in _failsafe_observe
    raise Exception(msg)
Exception: Giving up to connect to the gym duckietown server at host: evaluator
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
2849331Andrea Censi 🇨🇭Random executionaido1_LF1_r3-v3step4-vizsuccessnorudolf-30:01:10
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
No reset possible
2847331Andrea Censi 🇨🇭Random executionaido1_LF1_r3-v3step3-videossuccessnorudolf-30:00:48
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
No reset possible
2845331Andrea Censi 🇨🇭Random executionaido1_LF1_r3-v3step2-scoringsuccessnorudolf-30:01:44
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median9.066666666666643


other stats
episodes
details{"ep000": {"nsteps": 170, "reward": -7.8147632960010975, "good_angle": 24.324510145168993, "survival_time": 5.666666666666656, "traveled_tiles": 2, "valid_direction": 3.899999999999987}, "ep001": {"nsteps": 378, "reward": -3.7408004655409113, "good_angle": 23.919518238417485, "survival_time": 12.599999999999966, "traveled_tiles": 3, "valid_direction": 3.966666666666653}, "ep002": {"nsteps": 377, "reward": -2.72141292983952, "good_angle": 0.10818263866642196, "survival_time": 12.566666666666633, "traveled_tiles": 4, "valid_direction": 0}, "ep003": {"nsteps": 259, "reward": -6.280674806432835, "good_angle": 25.259055115670463, "survival_time": 8.633333333333313, "traveled_tiles": 3, "valid_direction": 3.9666666666666535}, "ep004": {"nsteps": 272, "reward": -5.42254740076689, "good_angle": 4.527672901365091, "survival_time": 9.066666666666643, "traveled_tiles": 3, "valid_direction": 3.1666666666666563}}
good_angle_max25.259055115670463
good_angle_mean15.627787807857692
good_angle_median23.919518238417485
good_angle_min0.10818263866642196
reward_max-2.72141292983952
reward_mean-5.19603977971625
reward_median-5.42254740076689
reward_min-7.8147632960010975
survival_time_max12.599999999999966
survival_time_mean9.706666666666644
survival_time_min5.666666666666656
traveled_tiles_max4
traveled_tiles_mean3
traveled_tiles_median3
traveled_tiles_min2
valid_direction_max3.9666666666666535
valid_direction_mean2.9999999999999902
valid_direction_median3.899999999999987
valid_direction_min0
No reset possible
2843332Andrea Censi 🇨🇭Random executionaido1_LFV_r1-v3step2-scoringsuccessnorudolf-30:00:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median5.666666666666656


other stats
episodes
details{"ep000": {"nsteps": 170, "reward": -7.721164818020428, "good_angle": 24.77476396957774, "survival_time": 5.666666666666656, "traveled_tiles": 2, "valid_direction": 3.933333333333321}, "ep001": {"nsteps": 357, "reward": -3.5732278295007407, "good_angle": 17.115294180815653, "survival_time": 11.899999999999968, "traveled_tiles": 3, "valid_direction": 3.93333333333332}, "ep002": {"nsteps": 113, "reward": -10.0194361566739, "good_angle": 4.400998063321564, "survival_time": 3.766666666666663, "traveled_tiles": 1, "valid_direction": 2.9666666666666632}, "ep003": {"nsteps": 42, "reward": -25.589645553202857, "good_angle": 0.04238394544622298, "survival_time": 1.400000000000001, "traveled_tiles": 1, "valid_direction": 0}, "ep004": {"nsteps": 500, "reward": -1.2579012894630432, "good_angle": 0.031289864774029244, "survival_time": 16.666666666666654, "traveled_tiles": 5, "valid_direction": 0}}
good_angle_max24.77476396957774
good_angle_mean9.272946004787045
good_angle_median4.400998063321564
good_angle_min0.031289864774029244
reward_max-1.2579012894630432
reward_mean-9.632275129372193
reward_median-7.721164818020428
reward_min-25.589645553202857
survival_time_max16.666666666666654
survival_time_mean7.879999999999988
survival_time_min1.400000000000001
traveled_tiles_max5
traveled_tiles_mean2.4
traveled_tiles_median2
traveled_tiles_min1
valid_direction_max3.933333333333321
valid_direction_mean2.1666666666666607
valid_direction_median2.9666666666666632
valid_direction_min0
No reset possible
2842332Andrea Censi 🇨🇭Random executionaido1_LFV_r1-v3step1-simulationsuccessnorudolf-30:02:03
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
No reset possible
2840330Benjamin Ramtoula 🇨🇦PyTorch DDPG templateaido1_LF1_r3-v3step1-simulationerrornorudolf-30:10:29
Timeout: Waited 603 [...]
Timeout:

Waited 603.045725107 for container to finish. Giving up.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
2837328Benjamin Ramtoula 🇨🇦PyTorch DDPG templateaido1_LF1_r2-v3step1-simulationerroryesrudolf-30:10:28
Timeout: Waited 602 [...]
Timeout:

Waited 602.97816515 for container to finish. Giving up.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
2835327Benjamin Ramtoula 🇨🇦PyTorch DDPG templateaido1_LF1_r2-v3step1-simulationerroryesrudolf-30:10:31
Timeout: Waited 602 [...]
Timeout:

Waited 602.899330139 for container to finish. Giving up.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
2828324Manfred DiazTensorflow templateaido1_LF1_r3-v3step1-simulationfailednorudolf-30:01:18
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 586, in wrap_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Uncaught exception in solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 581, in wrap_solution
    solution.run(cis)
  File "solution.py", line 64, in run
    solve(params, cis)  # let's try to solve the challenge, exciting ah?
  File "solution.py", line 40, in solve
    observation, reward, done, info = env.step(action)
  File "/usr/local/lib/python2.7/dist-packages/gym/wrappers/time_limit.py", line 31, in step
    observation, reward, done, info = self.env.step(action)
  File "/notebooks/src/gym-duckietown-agent/gym_duckietown_agent/envs/simplesimagent_env.py", line 107, in step
    obs, rew, done, misc = self.sim.step(action, with_observation=True)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 55, in step
    return self._failsafe_observe(msg)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 86, in _failsafe_observe
    raise Exception(msg)
Exception: Giving up to connect to the gym duckietown server at host: evaluator
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
2820194Florian GolemoPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationerrornorudolf-30:10:27
Timeout: Waited 602 [...]
Timeout:

Waited 602.795489788 for container to finish. Giving up.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
2817320Andrea Censi 🇨🇭PyTorch DDPG templateaido1_LF1_r3-v3step1-simulationfailednorudolf-30:00:35
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 586, in wrap_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Uncaught exception in solution:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 581, in wrap_solution
    solution.run(cis)
  File "solution.py", line 87, in run
    solve(params, cis)
  File "solution.py", line 60, in solve
    action = model.predict(observation)
  File "/workspace/model.py", line 176, in predict
    return self.actor(state).cpu().data.numpy().flatten()
  File "/opt/conda/lib/python2.7/site-packages/torch/nn/modules/module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)
  File "/workspace/model.py", line 72, in forward
    x = self.lr(self.lin1(x))
  File "/opt/conda/lib/python2.7/site-packages/torch/nn/modules/module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python2.7/site-packages/torch/nn/modules/linear.py", line 55, in forward
    return F.linear(input, self.weight, self.bias)
  File "/opt/conda/lib/python2.7/site-packages/torch/nn/functional.py", line 1024, in linear
    return torch.addmm(bias, input, weight.t())
RuntimeError: size mismatch, m1: [1 x 127872], m2: [4032 x 512] at /tmp/pip-req-build-vRRdPa/aten/src/TH/generic/THTensorMath.cpp:2070
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
2813149Julian ZillyBaseline solution using imitation learning from logsaido1_LF1_r3-v3step1-simulationerrornorudolf-30:00:09
Error while running [...]
Error while running Docker Compose:

Could not run ['docker-compose', '-p', 'job2813-5280', 'pull']:

   >  Command '['docker-compose', '-p', 'job2813-5280', 'pull']' returned non-zero exit status 1

 docker-compose stdout  | None
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible