Duckietown Challenges Home Challenges Submissions

Submission 14006

Submission14006
Competingyes
Challengeaido-LF-sim-validation
UserJacopo Tani
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobssim-0of4: 114672 sim-1of4: 114674 sim-2of4: 114673 sim-3of4: 114695
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

114695

Click the images to see detailed statistics about the episode.

loop-000

114674

Click the images to see detailed statistics about the episode.

zigzag-000

114673

Click the images to see detailed statistics about the episode.

autolab-000

114672

Click the images to see detailed statistics about the episode.

loop2-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
114695sim-3of4successyes0:04:03
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median6.922943789962439
survival_time_median59.99999999999873
deviation-center-line_median2.9242970428440547
in-drivable-lane_median4.29999999999978


other stats
deviation-center-line_max2.9242970428440547
deviation-center-line_mean2.9242970428440547
deviation-center-line_min2.9242970428440547
deviation-heading_max9.736682578145848
deviation-heading_mean9.736682578145848
deviation-heading_median9.736682578145848
deviation-heading_min9.736682578145848
distance-from-start_max2.4424228809379103
distance-from-start_mean2.4424228809379103
distance-from-start_median2.4424228809379103
distance-from-start_min2.4424228809379103
driven_any_max7.914378369472041
driven_any_mean7.914378369472041
driven_any_median7.914378369472041
driven_any_min7.914378369472041
driven_lanedir_consec_max6.922943789962439
driven_lanedir_consec_mean6.922943789962439
driven_lanedir_consec_min6.922943789962439
driven_lanedir_max7.066114134537271
driven_lanedir_mean7.066114134537271
driven_lanedir_median7.066114134537271
driven_lanedir_min7.066114134537271
in-drivable-lane_max4.29999999999978
in-drivable-lane_mean4.29999999999978
in-drivable-lane_min4.29999999999978
per-episodes
details{"loop-000-ego0": {"driven_any": 7.914378369472041, "survival_time": 59.99999999999873, "driven_lanedir": 7.066114134537271, "in-drivable-lane": 4.29999999999978, "deviation-heading": 9.736682578145848, "distance-from-start": 2.4424228809379103, "deviation-center-line": 2.9242970428440547, "driven_lanedir_consec": 6.922943789962439}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
114682sim-3of4successyes0:04:07
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
114674sim-1of4successyes0:05:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median6.187567934580509
survival_time_median59.99999999999873
deviation-center-line_median4.191806097585574
in-drivable-lane_median10.499999999999645


other stats
deviation-center-line_max4.191806097585574
deviation-center-line_mean4.191806097585574
deviation-center-line_min4.191806097585574
deviation-heading_max11.407249051295546
deviation-heading_mean11.407249051295546
deviation-heading_median11.407249051295546
deviation-heading_min11.407249051295546
distance-from-start_max4.369180363146929
distance-from-start_mean4.369180363146929
distance-from-start_median4.369180363146929
distance-from-start_min4.369180363146929
driven_any_max7.914461542552648
driven_any_mean7.914461542552648
driven_any_median7.914461542552648
driven_any_min7.914461542552648
driven_lanedir_consec_max6.187567934580509
driven_lanedir_consec_mean6.187567934580509
driven_lanedir_consec_min6.187567934580509
driven_lanedir_max6.187567934580509
driven_lanedir_mean6.187567934580509
driven_lanedir_median6.187567934580509
driven_lanedir_min6.187567934580509
in-drivable-lane_max10.499999999999645
in-drivable-lane_mean10.499999999999645
in-drivable-lane_min10.499999999999645
per-episodes
details{"zigzag-000-ego0": {"driven_any": 7.914461542552648, "survival_time": 59.99999999999873, "driven_lanedir": 6.187567934580509, "in-drivable-lane": 10.499999999999645, "deviation-heading": 11.407249051295546, "distance-from-start": 4.369180363146929, "deviation-center-line": 4.191806097585574, "driven_lanedir_consec": 6.187567934580509}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
114673sim-2of4successyes0:01:59
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.4037879349178603
survival_time_median11.65000000000003
deviation-center-line_median0.2290976016084681
in-drivable-lane_median8.000000000000036


other stats
deviation-center-line_max0.2290976016084681
deviation-center-line_mean0.2290976016084681
deviation-center-line_min0.2290976016084681
deviation-heading_max1.2137360708528802
deviation-heading_mean1.2137360708528802
deviation-heading_median1.2137360708528802
deviation-heading_min1.2137360708528802
distance-from-start_max1.011851724970244
distance-from-start_mean1.011851724970244
distance-from-start_median1.011851724970244
distance-from-start_min1.011851724970244
driven_any_max1.3953979440906643
driven_any_mean1.3953979440906643
driven_any_median1.3953979440906643
driven_any_min1.3953979440906643
driven_lanedir_consec_max0.4037879349178603
driven_lanedir_consec_mean0.4037879349178603
driven_lanedir_consec_min0.4037879349178603
driven_lanedir_max0.4037879349178603
driven_lanedir_mean0.4037879349178603
driven_lanedir_median0.4037879349178603
driven_lanedir_min0.4037879349178603
in-drivable-lane_max8.000000000000036
in-drivable-lane_mean8.000000000000036
in-drivable-lane_min8.000000000000036
per-episodes
details{"autolab-000-ego0": {"driven_any": 1.3953979440906643, "survival_time": 11.65000000000003, "driven_lanedir": 0.4037879349178603, "in-drivable-lane": 8.000000000000036, "deviation-heading": 1.2137360708528802, "distance-from-start": 1.011851724970244, "deviation-center-line": 0.2290976016084681, "driven_lanedir_consec": 0.4037879349178603}}
simulation-passed1
survival_time_max11.65000000000003
survival_time_mean11.65000000000003
survival_time_min11.65000000000003
No reset possible
114672sim-0of4successyes0:04:39
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median4.811518735737607
survival_time_median59.99999999999873
deviation-center-line_median5.08831365918317
in-drivable-lane_median22.14999999999958


other stats
deviation-center-line_max5.08831365918317
deviation-center-line_mean5.08831365918317
deviation-center-line_min5.08831365918317
deviation-heading_max6.784801939053964
deviation-heading_mean6.784801939053964
deviation-heading_median6.784801939053964
deviation-heading_min6.784801939053964
distance-from-start_max0.9608449157935064
distance-from-start_mean0.9608449157935064
distance-from-start_median0.9608449157935064
distance-from-start_min0.9608449157935064
driven_any_max7.914176863358285
driven_any_mean7.914176863358285
driven_any_median7.914176863358285
driven_any_min7.914176863358285
driven_lanedir_consec_max4.811518735737607
driven_lanedir_consec_mean4.811518735737607
driven_lanedir_consec_min4.811518735737607
driven_lanedir_max4.811518735737607
driven_lanedir_mean4.811518735737607
driven_lanedir_median4.811518735737607
driven_lanedir_min4.811518735737607
in-drivable-lane_max22.14999999999958
in-drivable-lane_mean22.14999999999958
in-drivable-lane_min22.14999999999958
per-episodes
details{"loop2-000-ego0": {"driven_any": 7.914176863358285, "survival_time": 59.99999999999873, "driven_lanedir": 4.811518735737607, "in-drivable-lane": 22.14999999999958, "deviation-heading": 6.784801939053964, "distance-from-start": 0.9608449157935064, "deviation-center-line": 5.08831365918317, "driven_lanedir_consec": 4.811518735737607}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
94809sim-2of4successno0:08:44
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
94807sim-3of4successno0:03:36
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
94806sim-0of4successno0:08:36
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
94801sim-1of4successno0:07:52
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
74766sim-0of4successno0:09:03
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
74765sim-2of4successno0:02:18
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
74763sim-1of4successno0:02:05
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
74761sim-3of4successno0:08:33
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
63260sim-2of4successno0:02:23
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
63258sim-1of4successno0:07:33
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
63257sim-0of4successno0:08:32
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
63256sim-3of4successno0:07:27
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
63253sim-2of4host-errorno0:01:14
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 83, in run_docker
    return subprocess.check_output(cmd0, cwd=cwd, stderr=sys.stderr)
  File "/usr/lib/python3.8/subprocess.py", line 411, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/lib/python3.8/subprocess.py", line 512, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['docker-compose', '-p', 'nogpu-production-b-spot-1-04_18237db0a2cd-job63253-509955', 'ps', '-q', 'evaluator', 'simulator', 'solution-ego0']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 42, in get_services_id
    output = run_docker(wd, project, cmd, get_output=True)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 90, in run_docker
    raise DockerComposeFail(
duckietown_challenges_runner.docker_compose.DockerComposeFail: Could not run command
│    cmd: [docker-compose, -p, nogpu-production-b-spot-1-04_18237db0a2cd-job63253-509955, ps, -q,
│          evaluator, simulator, solution-ego0]
│ stdout: ''
│  sderr: ''
│      e: Command '['docker-compose', '-p', 'nogpu-production-b-spot-1-04_18237db0a2cd-job63253-509955', 'ps', '-q', 'evaluator', 'simulator', 'solution-ego0']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 745, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 959, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 120, in write_logs
    services2id: Dict[ServiceName, ContainerID] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 63, in get_services_id
    raise DockerComposeFail(msg, output=output.decode(), names=names) from e
duckietown_challenges_runner.docker_compose.DockerComposeFail: Cannot get process ids
│ output: not available
│  names: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
63010347failedyes0:03:04
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 190, in main
    raise InvalidSubmission(msg)
duckietown_challenges.exceptions.InvalidSubmission: Timeout during connection to ego0: <SignalTimeout in state: 2>
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
63009347failedyes0:03:25
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 190, in main
    raise InvalidSubmission(msg)
duckietown_challenges.exceptions.InvalidSubmission: Timeout during connection to ego0: <SignalTimeout in state: 2>
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible