Duckietown Challenges Home Challenges Submissions

Submission 464

Submission464
Competingyes
Challengeaido1_LF1_r3-v3
UserOrlando Marquez 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💔
Jobsstep1-simulation: 13711 step2-scoring: 10276 step3-videos: 10280 step4-viz: 10307
Next
User labelPyTorch DDPG template
Admin priority50
Blessingn/a
User priority50

10307

Click the images to see detailed statistics about the episode.

ep000

ep001

ep002

ep003

ep004

10280

ep000

ep001

ep002

ep003

ep004

Evaluation jobs for this submission

See previous jobs for previous versions of challenges
Job IDstepstatusup to datedate starteddate completeddurationmessage
13711step1-simulationfailedyes0:07:17
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 590, in wrap_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Uncaught exception in solution:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 585, in wrap_solution
    solution.run(cis)
  File "solution.py", line 105, in run
    solve(params, cis)
  File "solution.py", line 70, in solve
    observation, reward, done, info = env.step(action)
  File "/opt/conda/lib/python2.7/site-packages/gym/core.py", line 332, in step
    return self.env.step(action)
  File "/opt/conda/lib/python2.7/site-packages/gym/core.py", line 332, in step
    return self.env.step(action)
  File "/opt/conda/lib/python2.7/site-packages/gym/core.py", line 304, in step
    observation, reward, done, info = self.env.step(action)
  File "/opt/conda/lib/python2.7/site-packages/gym/core.py", line 304, in step
    observation, reward, done, info = self.env.step(action)
  File "/opt/conda/lib/python2.7/site-packages/gym/core.py", line 304, in step
    observation, reward, done, info = self.env.step(action)
  File "/opt/conda/lib/python2.7/site-packages/gym/wrappers/time_limit.py", line 31, in step
    observation, reward, done, info = self.env.step(action)
  File "/workspace/src/gym-duckietown-agent/gym_duckietown_agent/envs/simplesimagent_env.py", line 109, in step
    obs, rew, done, misc = self.sim.step(action, with_observation=True)
  File "/workspace/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 55, in step
    return self._failsafe_observe(msg)
  File "/workspace/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 86, in _failsafe_observe
    raise Exception(msg)
Exception: Giving up to connect to the gym duckietown server at host: evaluator


Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
10307step4-vizsuccessyes0:06:29
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_median3.100249689175728
deviation-center-line_median1.3449474516272864
in-drivable-lane_median1.6999999999999946


other stats
deviation-center-line_max1.3768982623538213
deviation-center-line_mean1.1562875543915694
deviation-center-line_min0.3685860053560791
deviation-heading_max11.68388141943642
deviation-heading_mean9.352449839692795
deviation-heading_median10.618271797823088
deviation-heading_min3.660740042655455
driven_any_max4.979113636122271
driven_any_mean4.852610389758444
driven_any_median4.979031772314665
driven_any_min4.346832205322924
driven_lanedir_max3.167659217109948
driven_lanedir_mean2.624352949811872
driven_lanedir_min0.676559070177116
in-drivable-lane_max10.433333333333302
in-drivable-lane_mean3.339999999999989
in-drivable-lane_min0.8666666666666638
per-episodes
details{"ep000": {"driven_any": 4.346832205322924, "driven_lanedir": 0.676559070177116, "in-drivable-lane": 10.433333333333302, "deviation-heading": 3.660740042655455, "deviation-center-line": 0.3685860053560791}, "ep001": {"driven_any": 4.979113636122251, "driven_lanedir": 3.05494506528675, "in-drivable-lane": 2.199999999999993, "deviation-heading": 9.953043798180028, "deviation-center-line": 1.3768982623538213}, "ep002": {"driven_any": 4.979031772314665, "driven_lanedir": 3.1223517073098175, "in-drivable-lane": 1.4999999999999951, "deviation-heading": 10.846312140368983, "deviation-center-line": 1.3449474516272864}, "ep003": {"driven_any": 4.978960698910106, "driven_lanedir": 3.167659217109948, "in-drivable-lane": 0.8666666666666638, "deviation-heading": 11.68388141943642, "deviation-center-line": 1.342293455826984}, "ep004": {"driven_any": 4.979113636122271, "driven_lanedir": 3.100249689175728, "in-drivable-lane": 1.6999999999999946, "deviation-heading": 10.618271797823088, "deviation-center-line": 1.3487125967936766}}
No reset possible
10306step4-vizsuccessyes0:06:27
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
10305step4-vizsuccessyes0:06:31
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
10304step4-vizsuccessyes0:06:29
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
10303step4-vizerroryes0:16:25
Timeout: Waited 601 [...]
Timeout:

Waited 601.376960039 for container to finish. Giving up. 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
10280step3-videossuccessyes0:04:02
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
No reset possible
10276step2-scoringsuccessyes0:00:23
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median16.666666666666654


other stats
episodes
details{"ep000": {"nsteps": 437, "reward": -2.9288726860958496, "good_angle": 23.9779702734845, "survival_time": 14.566666666666626, "traveled_tiles": 1, "valid_direction": 10.433333333333303}, "ep001": {"nsteps": 500, "reward": -0.3779372270097956, "good_angle": 13.49085374796333, "survival_time": 16.666666666666654, "traveled_tiles": 2, "valid_direction": 12.96666666666665}, "ep002": {"nsteps": 500, "reward": -0.25310920631792394, "good_angle": 13.56356330463625, "survival_time": 16.666666666666654, "traveled_tiles": 2, "valid_direction": 12.866666666666656}, "ep003": {"nsteps": 500, "reward": -0.1725637492822534, "good_angle": 13.543601805681016, "survival_time": 16.666666666666654, "traveled_tiles": 1, "valid_direction": 12.999999999999986}, "ep004": {"nsteps": 500, "reward": -0.3079843127221684, "good_angle": 13.764277431711864, "survival_time": 16.666666666666654, "traveled_tiles": 2, "valid_direction": 13.033333333333324}}
good_angle_max23.9779702734845
good_angle_mean15.668053312695392
good_angle_median13.56356330463625
good_angle_min13.49085374796333
reward_max-0.1725637492822534
reward_mean-0.808093436285598
reward_median-0.3079843127221684
reward_min-2.9288726860958496
survival_time_max16.666666666666654
survival_time_mean16.24666666666665
survival_time_min14.566666666666626
traveled_tiles_max2
traveled_tiles_mean1.6
traveled_tiles_median2
traveled_tiles_min1
valid_direction_max13.033333333333324
valid_direction_mean12.459999999999985
valid_direction_median12.96666666666665
valid_direction_min10.433333333333303
No reset possible