Evaluator 4960

aido5-LFV_multi-real-validation

2020-12-12 10:17:04+00:00

2020-12-12 10:21:48+00:00

0:04:44

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

driven_lanedir_consec	1.3674558658897027
survival_time	22.798027515411377
deviation-center-line	0.946841898300072
in-drivable-lane	3.0018957455952964

other stats

deviation-heading	4.186793112337633
driven_any	1.648105784346581
driven_lanedir	1.3674558658897027
visualized-eval0-passed	1

No reset possible

61086

13309

Robert Moni 🇭🇺

speedRL

eval0-visualize

host-error

2020-12-12 10:12:01+00:00

2020-12-12 10:13:59+00:00

0:01:58

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 53, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 745, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 959, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 120, in write_logs
    services2id: Dict[ServiceName, ContainerID] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 63, in get_services_id
    raise DockerComposeFail(msg, output=output.decode(), names=names) from e
duckietown_challenges_runner.docker_compose.DockerComposeFail: Cannot get process ids
│ output: ''
│  names: {}

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

61073

exercise_state_estimation

eval1-visualize

host-error

aido5-LF-real-validation

2020-12-12 10:10:11+00:00

2020-12-12 10:11:22+00:00

0:01:11

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 53, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 745, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 959, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 120, in write_logs
    services2id: Dict[ServiceName, ContainerID] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 63, in get_services_id
    raise DockerComposeFail(msg, output=output.decode(), names=names) from e
duckietown_challenges_runner.docker_compose.DockerComposeFail: Cannot get process ids
│ output: ''
│  names: {}

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

60981

12909

Robert Moni 🇭🇺

speedRL

eval1-visualize

success

2020-12-12 06:59:14+00:00

2020-12-12 07:01:27+00:00

0:02:13

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

driven_lanedir_consec	0.19787301936556775
survival_time	6.201350688934326
deviation-center-line	0.3035517867644619
in-drivable-lane	4.020295858383179

other stats

deviation-heading	1.1787668532802396
driven_any	0.5150670563716394
driven_lanedir	0.19787301936556775
visualized-eval1-passed	1

No reset possible

60925

exercise_state_estimation

eval0-videos

host-error

2020-12-12 02:06:14+00:00

2020-12-12 02:07:30+00:00

0:01:16

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 83, in run_docker
    return subprocess.check_output(cmd0, cwd=cwd, stderr=sys.stderr)
  File "/usr/lib/python3.8/subprocess.py", line 411, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/lib/python3.8/subprocess.py", line 512, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['docker-compose', '-p', 'nogpu-production-2-05_e97ecf26b6b4-job60925-459000', 'ps', '-q', 'evaluator']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 42, in get_services_id
    output = run_docker(wd, project, cmd, get_output=True)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 90, in run_docker
    raise DockerComposeFail(
duckietown_challenges_runner.docker_compose.DockerComposeFail: Could not run command
│    cmd: [docker-compose, -p, nogpu-production-2-05_e97ecf26b6b4-job60925-459000, ps, -q,
│          evaluator]
│ stdout: ''
│  sderr: ''
│      e: Command '['docker-compose', '-p', 'nogpu-production-2-05_e97ecf26b6b4-job60925-459000', 'ps', '-q', 'evaluator']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 745, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 959, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 120, in write_logs
    services2id: Dict[ServiceName, ContainerID] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 63, in get_services_id
    raise DockerComposeFail(msg, output=output.decode(), names=names) from e
duckietown_challenges_runner.docker_compose.DockerComposeFail: Cannot get process ids
│ output: not available
│  names: {}

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

60851

13289

Bea Baselines 🐤

baseline-duckietown

eval0-visualize

host-error

aido5-LF-real-validation

2020-12-12 01:55:01+00:00

2020-12-12 01:56:07+00:00

0:01:06

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 53, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 745, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 959, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 120, in write_logs
    services2id: Dict[ServiceName, ContainerID] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 63, in get_services_id
    raise DockerComposeFail(msg, output=output.decode(), names=names) from e
duckietown_challenges_runner.docker_compose.DockerComposeFail: Cannot get process ids
│ output: ''
│  names: {}

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

60807

13284

András Kalapos 🇭🇺

real-v1.0-3092-363

eval2-visualize

error

2020-12-12 01:29:58+00:00

2020-12-12 01:30:49+00:00

0:00:51

InvalidEvaluator: Tr [...]

InvalidEvaluator:
Traceback (most recent call last):
  File "main.py", line 19, in wrap
    stats = evaluate_robotarium_log(d_step, outd=outd)
  File "/project/evaluate_robotarium_log.py", line 133, in evaluate_robotarium_log
    pose = seq_raw.values[0]
IndexError: list index out of range

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "main.py", line 34, in <module>
    wrap(_)
  File "main.py", line 24, in wrap
    raise dc.InvalidEvaluator() from e
duckietown_challenges.exceptions.InvalidEvaluator:

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

60720

13304

Raphael Jean

exercise_state_estimation

eval0-visualize

host-error

2020-12-12 01:03:32+00:00

2020-12-12 01:04:44+00:00

0:01:12

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 53, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 745, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 959, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 120, in write_logs
    services2id: Dict[ServiceName, ContainerID] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 63, in get_services_id
    raise DockerComposeFail(msg, output=output.decode(), names=names) from e
duckietown_challenges_runner.docker_compose.DockerComposeFail: Cannot get process ids
│ output: ''
│  names: {}

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

60709

13289

Bea Baselines 🐤

baseline-duckietown

eval0-visualize

error

2020-12-12 01:01:42+00:00

2020-12-12 01:03:26+00:00

0:01:44

InvalidEvaluator: Tr [...]

InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/registrar.py", line 44, in instance
    block = generator(name=name, config=config, library=library)
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model_loader.py", line 57, in __call__
    model = create_from_parsing_results(parsed_model, name, config, library=sandbox)
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model_instantiation.py", line 312, in create_from_parsing_results
    raise SemanticError(msg, element)
procgraph.core.exceptions.SemanticError: Could not find block type 'bagread' in 164 options. (Did you mean 'bagread_test'?)
In file /project/evaluate_robotarium_log.py:
 line 12 >config t1     """ Time end, relative to physical bag initial timestamp """
 line 13 >
 line 14 >|bagread file=$bag topics=$topic t0=$t0 t1=$t1| --> cam
          ^
          |
          here or nearby

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "main.py", line 19, in wrap
    stats = evaluate_robotarium_log(d_step, outd=outd)
  File "/project/evaluate_robotarium_log.py", line 172, in evaluate_robotarium_log
    make_video(dirname=dirname, seq=closest, outd=outd, robot_name=robot_name)
  File "/project/evaluate_robotarium_log.py", line 294, in make_video
    make_video_cut(dirname, chunk.wt, chunk.t0, chunk.t1, outd, outfile)
  File "/project/evaluate_robotarium_log.py", line 345, in make_video_cut
    pg('bag2mp4_fixfps_limit_robotarium', dict(bag=bag_filename, topic=topic, t0=t0use, t1=t1use, out=out))
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 245, in pg
    model = library.instance(filename, name="cmdline", config=config)
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/registrar.py", line 56, in instance
    raise ModelInstantionError(msg) from e  # TODO: use Procgraph own exception
procgraph.core.exceptions.ModelInstantionError: Could not instance block from generator.
        name: cmdline  
      config: {'bag': '/challenges/previous-steps/eval0/watchtowers/watchtower128/watchtower.bag', 'topic': '/watchtower128/camera_node/image/compressed', 't0': 11.451447010040283, 't1': 15.45258355140686, 'out': '/challenges/tmp/tmpf_joqdcw/visualization/tmp/ego0-0-watchtower128-00000-00400.mp4'}  
   generator: <procgraph.core.model_loader.ModelSpec object at 0x7f1d68515160>  
     of type: <class 'procgraph.core.model_loader.ModelSpec'>  
Because of this exception:
| Could not find block type 'bagread' in 164 options. (Did you mean 'bagread_test'?)
| In file /project/evaluate_robotarium_log.py:
|  line 12 >config t1     """ Time end, relative to physical bag initial timestamp """
|  line 13 >
|  line 14 >|bagread file=$bag topics=$topic t0=$t0 t1=$t1| --> cam
|           ^
|           |
|           here or nearby

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "main.py", line 34, in <module>
    wrap(_)
  File "main.py", line 24, in wrap
    raise dc.InvalidEvaluator() from e
duckietown_challenges.exceptions.InvalidEvaluator:

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

60700

13289

Bea Baselines 🐤

baseline-duckietown

eval0-videos

host-error

2020-12-12 00:58:23+00:00

2020-12-12 00:59:29+00:00

0:01:06

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 53, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 745, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 959, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 120, in write_logs
    services2id: Dict[ServiceName, ContainerID] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 63, in get_services_id
    raise DockerComposeFail(msg, output=output.decode(), names=names) from e
duckietown_challenges_runner.docker_compose.DockerComposeFail: Cannot get process ids
│ output: ''
│  names: {}

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

60671

13304

Raphael Jean

exercise_state_estimation

eval0-videos

host-error

2020-12-12 00:54:31+00:00

2020-12-12 00:55:56+00:00

0:01:25

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 53, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 745, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 959, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 120, in write_logs
    services2id: Dict[ServiceName, ContainerID] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 63, in get_services_id
    raise DockerComposeFail(msg, output=output.decode(), names=names) from e
duckietown_challenges_runner.docker_compose.DockerComposeFail: Cannot get process ids
│ output: ''
│  names: {}

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

60666

exercise_state_estimation

eval1-videos

host-error

2020-12-12 00:53:15+00:00

2020-12-12 00:54:24+00:00

0:01:09

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 53, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 745, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 959, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 120, in write_logs
    services2id: Dict[ServiceName, ContainerID] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 63, in get_services_id
    raise DockerComposeFail(msg, output=output.decode(), names=names) from e
duckietown_challenges_runner.docker_compose.DockerComposeFail: Cannot get process ids
│ output: ''
│  names: {}

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

60655

exercise_state_estimation

eval0-visualize

host-error

aido5-LF-real-validation

2020-12-12 00:51:48+00:00

2020-12-12 00:52:56+00:00

0:01:08

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 53, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 745, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 959, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 120, in write_logs
    services2id: Dict[ServiceName, ContainerID] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 63, in get_services_id
    raise DockerComposeFail(msg, output=output.decode(), names=names) from e
duckietown_challenges_runner.docker_compose.DockerComposeFail: Cannot get process ids
│ output: ''
│  names: {}

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

60641

13284

András Kalapos 🇭🇺

real-v1.0-3092-363

eval2-visualize

error

2020-12-12 00:47:50+00:00

2020-12-12 00:49:25+00:00

0:01:35

InvalidEvaluator: Tr [...]

InvalidEvaluator:
Traceback (most recent call last):
  File "main.py", line 19, in wrap
    stats = evaluate_robotarium_log(d_step, outd=outd)
  File "/project/evaluate_robotarium_log.py", line 133, in evaluate_robotarium_log
    pose = seq_raw.values[0]
IndexError: list index out of range

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "main.py", line 34, in <module>
    wrap(_)
  File "main.py", line 24, in wrap
    raise dc.InvalidEvaluator() from e
duckietown_challenges.exceptions.InvalidEvaluator:

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

60620

13304

Raphael Jean

exercise_state_estimation

eval1-videos

success