Submission 17120

Submission

17120

Competing

yes

Challenge

aido-hello-sim-validation

User

Áron Tibor Tóth

Date submitted

2021-10-04 11:19:01+00:00

Last status update

2022-01-25 14:02:07+00:00

Complete

complete

Details

Evaluation is complete.

Sisters

Result

💚

Jobs

sanity-check: 128543

User label

template-random

Admin priority

Blessing

n/a

User priority

128543

Click the images to see detailed statistics about the episode.

scenarios

Evaluation jobs for this submission

Show only up-to-date jobs

Job ID

step

status

up to date

date started

date completed

duration

message

128543

sanity-check

success

yes

2022-01-25 13:53:22+00:00

2022-01-25 13:55:22+00:00

0:02:00

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

survival_time_median	0.49999999999999994
in-drivable-lane_median	0.0
driven_lanedir_consec_median	0.058817380809317754
deviation-center-line_median	0.006975964128268102

other stats

deviation-center-line_max	0.006975964128268102
deviation-center-line_mean	0.006975964128268102
deviation-center-line_min	0.006975964128268102
deviation-heading_max	0.2650181877737848
deviation-heading_mean	0.2650181877737848
deviation-heading_median	0.2650181877737848
deviation-heading_min	0.2650181877737848
distance-from-start_max	0.06765951470550984
distance-from-start_mean	0.06765951470550984
distance-from-start_median	0.06765951470550984
distance-from-start_min	0.06765951470550984
driven_any_max	0.0676644346818848
driven_any_mean	0.0676644346818848
driven_any_median	0.0676644346818848
driven_any_min	0.0676644346818848
driven_lanedir_consec_max	0.058817380809317754
driven_lanedir_consec_mean	0.058817380809317754
driven_lanedir_consec_min	0.058817380809317754
driven_lanedir_max	0.058817380809317754
driven_lanedir_mean	0.058817380809317754
driven_lanedir_median	0.058817380809317754
driven_lanedir_min	0.058817380809317754
in-drivable-lane_max	0.0
in-drivable-lane_mean	0.0
in-drivable-lane_min	0.0
per-episodes	details {"scenarios-ego0": {"driven_any": 0.0676644346818848, "survival_time": 0.49999999999999994, "driven_lanedir": 0.058817380809317754, "in-drivable-lane": 0.0, "deviation-heading": 0.2650181877737848, "distance-from-start": 0.06765951470550984, "deviation-center-line": 0.006975964128268102, "driven_lanedir_consec": 0.058817380809317754}}
simulation-passed	1
survival_time_max	0.49999999999999994
survival_time_mean	0.49999999999999994
survival_time_min	0.49999999999999994

No reset possible

128535

sanity-check

success

yes

2022-01-25 13:51:44+00:00

2022-01-25 13:52:45+00:00

0:01:01

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

128529

sanity-check

success

yes

2022-01-25 13:50:08+00:00

2022-01-25 13:51:10+00:00

0:01:02

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

128526

sanity-check

success

yes

2022-01-25 13:48:53+00:00

2022-01-25 13:49:59+00:00

0:01:06

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

107859

sanity-check

failed

2021-11-03 13:59:14+00:00

2021-11-03 13:59:50+00:00

0:00:36

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

107853

sanity-check

failed

2021-11-03 13:58:15+00:00

2021-11-03 13:58:50+00:00

0:00:35

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

107851

sanity-check

failed

2021-11-03 13:57:12+00:00

2021-11-03 13:57:48+00:00

0:00:36

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

107846

sanity-check

failed

2021-11-03 13:56:08+00:00

2021-11-03 13:56:45+00:00

0:00:37

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

107843

sanity-check

failed

2021-11-03 13:55:14+00:00

2021-11-03 13:55:50+00:00

0:00:36

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

107835

sanity-check

failed

2021-11-03 13:53:59+00:00

2021-11-03 13:54:34+00:00

0:00:35

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

107830

sanity-check

failed

2021-11-03 13:53:11+00:00

2021-11-03 13:53:41+00:00

0:00:30

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

107823

sanity-check

failed

2021-11-03 13:51:59+00:00

2021-11-03 13:52:40+00:00

0:00:41

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

88902

370

success

yes

2021-10-19 11:47:23+00:00

2021-10-19 11:49:12+00:00

0:01:49

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

86197

370

error

2021-10-04 23:12:50+00:00

2021-10-04 23:14:12+00:00

0:01:22

The result file [...]

    The result file is not found in working dir /home/aido/duckietown-tmp-evals/production/aido-hello-sim-validation/submission17120/LFv-sim-montreal03_a39f3a5a13b9-job86197-a-wd:

    File '/home/aido/duckietown-tmp-evals/production/aido-hello-sim-validation/submission17120/LFv-sim-montreal03_a39f3a5a13b9-job86197-a-wd/challenge-results/challenge_results.yaml' does not exist.

    This usually means that the evaluator did not finish and some times that there was an import error.
    Check the evaluator log to see what happened.

List of all files:

 -/home/aido/duckietown-tmp-evals/production/aido-hello-sim-validation/submission17120/LFv-sim-montreal03_a39f3a5a13b9-job86197-a-wd/docker-compose.original.yaml
-/home/aido/duckietown-tmp-evals/production/aido-hello-sim-validation/submission17120/LFv-sim-montreal03_a39f3a5a13b9-job86197-a-wd/runner
-/home/aido/duckietown-tmp-evals/production/aido-hello-sim-validation/submission17120/LFv-sim-montreal03_a39f3a5a13b9-job86197-a-wd/docker-compose.yaml
-/home/aido/duckietown-tmp-evals/production/aido-hello-sim-validation/submission17120/LFv-sim-montreal03_a39f3a5a13b9-job86197-a-wd/logs/evaluator/online.log
-/home/aido/duckietown-tmp-evals/production/aido-hello-sim-validation/submission17120/LFv-sim-montreal03_a39f3a5a13b9-job86197-a-wd/logs/evaluator/online.html
-/home/aido/duckietown-tmp-evals/production/aido-hello-sim-validation/submission17120/LFv-sim-montreal03_a39f3a5a13b9-job86197-a-wd/logs/simulator/online.log
-/home/aido/duckietown-tmp-evals/production/aido-hello-sim-validation/submission17120/LFv-sim-montreal03_a39f3a5a13b9-job86197-a-wd/logs/simulator/online.html
-/home/aido/duckietown-tmp-evals/production/aido-hello-sim-validation/submission17120/LFv-sim-montreal03_a39f3a5a13b9-job86197-a-wd/logs/challenges-runner/stdout.log
-/home/aido/duckietown-tmp-evals/production/aido-hello-sim-validation/submission17120/LFv-sim-montreal03_a39f3a5a13b9-job86197-a-wd/logs/challenges-runner/stderr.log
-/home/aido/duckietown-tmp-evals/production/aido-hello-sim-validation/submission17120/LFv-sim-montreal03_a39f3a5a13b9-job86197-a-wd/logs/solution-ego0/online.log
-/home/aido/duckietown-tmp-evals/production/aido-hello-sim-validation/submission17120/LFv-sim-montreal03_a39f3a5a13b9-job86197-a-wd/logs/solution-ego0/online.html

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible