Duckietown Challenges Home Challenges Submissions

Submission 9361

Submission9361
Competingyes
Challengeaido5-LF-sim-validation
UserFrancois Hebert
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 58180
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

58180

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
58180LFv-simsuccessyes0:20:48
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median3.726019554029402
survival_time_median36.22499999999938
deviation-center-line_median1.1615994432038133
in-drivable-lane_median3.900000000000021


other stats
agent_compute-ego0_max0.01300727795006135
agent_compute-ego0_mean0.012540059754404906
agent_compute-ego0_median0.012518001503414578
agent_compute-ego0_min0.012116958060729117
complete-iteration_max0.2295787849544007
complete-iteration_mean0.19008980618047816
complete-iteration_median0.18482287331088
complete-iteration_min0.16113469314575196
deviation-center-line_max2.8227376355564613
deviation-center-line_mean1.3394711224423406
deviation-center-line_min0.21194796780527408
deviation-heading_max7.346939029046007
deviation-heading_mean4.0468688509161
deviation-heading_median3.848223958388897
deviation-heading_min1.1440884578406003
driven_any_max12.090169106254123
driven_any_mean6.975713687541877
driven_any_median7.196062355016798
driven_any_min1.420560933879784
driven_lanedir_consec_max11.485006601942532
driven_lanedir_consec_mean4.9203273753061865
driven_lanedir_consec_min0.7442637912234091
driven_lanedir_max11.485006601942532
driven_lanedir_mean4.9203273753061865
driven_lanedir_median3.726019554029402
driven_lanedir_min0.7442637912234091
get_duckie_state_max1.3666152954101563e-06
get_duckie_state_mean1.330794928893218e-06
get_duckie_state_median1.322283713294419e-06
get_duckie_state_min1.3119969935738772e-06
get_robot_state_max0.003841259199614132
get_robot_state_mean0.003762859032857739
get_robot_state_median0.0037709089267401047
get_robot_state_min0.003668359078336616
get_state_dump_max0.004815171898453559
get_state_dump_mean0.004697948039131306
get_state_dump_median0.0046879596356992365
get_state_dump_min0.004600700986673195
get_ui_image_max0.036020143532458644
get_ui_image_mean0.03088353885890727
get_ui_image_median0.03050260331410353
get_ui_image_min0.02650880527496338
in-drivable-lane_max29.59999999999925
in-drivable-lane_mean9.862499999999802
in-drivable-lane_min2.049999999999912
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 12.090169106254123, "get_ui_image": 0.027621951528036228, "step_physics": 0.09963775614119888, "survival_time": 59.99999999999873, "driven_lanedir": 11.485006601942532, "get_state_dump": 0.004600700986673195, "get_robot_state": 0.003668359078336616, "sim_render-ego0": 0.0037419232202509263, "get_duckie_state": 1.3185480452894073e-06, "in-drivable-lane": 2.049999999999912, "deviation-heading": 7.346939029046007, "agent_compute-ego0": 0.012116958060729117, "complete-iteration": 0.16493285605552097, "set_robot_commands": 0.002206753930084711, "deviation-center-line": 2.8227376355564613, "driven_lanedir_consec": 11.485006601942532, "sim_compute_sim_state": 0.009243286420264708, "sim_compute_performance-ego0": 0.00200957422153241}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.420560933879784, "get_ui_image": 0.036020143532458644, "step_physics": 0.15367228896529586, "survival_time": 8.04999999999998, "driven_lanedir": 0.7442637912234091, "get_state_dump": 0.004659701276708532, "get_robot_state": 0.003735293576746811, "sim_render-ego0": 0.0038731598559721015, "get_duckie_state": 1.326019381299431e-06, "in-drivable-lane": 3.8499999999999863, "deviation-heading": 1.1440884578406003, "agent_compute-ego0": 0.012717975510491263, "complete-iteration": 0.2295787849544007, "set_robot_commands": 0.0022787532688658915, "deviation-center-line": 0.21194796780527408, "driven_lanedir_consec": 0.7442637912234091, "sim_compute_sim_state": 0.010433322117652424, "sim_compute_performance-ego0": 0.0020986692405041353}, "LF-norm-techtrack-000-ego0": {"driven_any": 12.07987401354121, "get_ui_image": 0.03338325510017084, "step_physics": 0.1268679251976553, "survival_time": 59.99999999999873, "driven_lanedir": 5.837279608330068, "get_state_dump": 0.004815171898453559, "get_robot_state": 0.003841259199614132, "sim_render-ego0": 0.003929315459023507, "get_duckie_state": 1.3119969935738772e-06, "in-drivable-lane": 29.59999999999925, "deviation-heading": 6.313710685467765, "agent_compute-ego0": 0.01300727795006135, "complete-iteration": 0.20471289056623904, "set_robot_commands": 0.0023124279527243327, "deviation-center-line": 1.8908444143900205, "driven_lanedir_consec": 5.837279608330068, "sim_compute_sim_state": 0.014390997247433882, "sim_compute_performance-ego0": 0.0020741654077636312}, "LF-norm-small_loop-000-ego0": {"driven_any": 2.312250696492387, "get_ui_image": 0.02650880527496338, "step_physics": 0.09939010143280028, "survival_time": 12.450000000000042, "driven_lanedir": 1.6147594997287356, "get_state_dump": 0.0047162179946899415, "get_robot_state": 0.0038065242767333983, "sim_render-ego0": 0.00394497013092041, "get_duckie_state": 1.3666152954101563e-06, "in-drivable-lane": 3.950000000000056, "deviation-heading": 1.3827372313100297, "agent_compute-ego0": 0.012318027496337891, "complete-iteration": 0.16113469314575196, "set_robot_commands": 0.002512011528015137, "deviation-center-line": 0.4323544720176062, "driven_lanedir_consec": 1.6147594997287356, "sim_compute_sim_state": 0.0057320261001586915, "sim_compute_performance-ego0": 0.0021183061599731445}}
set_robot_commands_max0.002512011528015137
set_robot_commands_mean0.0023274866699225178
set_robot_commands_median0.002295590610795112
set_robot_commands_min0.002206753930084711
sim_compute_performance-ego0_max0.0021183061599731445
sim_compute_performance-ego0_mean0.0020751787574433303
sim_compute_performance-ego0_median0.002086417324133883
sim_compute_performance-ego0_min0.00200957422153241
sim_compute_sim_state_max0.014390997247433882
sim_compute_sim_state_mean0.009949907971377427
sim_compute_sim_state_median0.009838304268958566
sim_compute_sim_state_min0.0057320261001586915
sim_render-ego0_max0.00394497013092041
sim_render-ego0_mean0.003872342166541736
sim_render-ego0_median0.003901237657497804
sim_render-ego0_min0.0037419232202509263
simulation-passed1
step_physics_max0.15367228896529586
step_physics_mean0.11989201793423758
step_physics_median0.11325284066942708
step_physics_min0.09939010143280028
survival_time_max59.99999999999873
survival_time_mean35.12499999999937
survival_time_min8.04999999999998
No reset possible
58177LFv-simhost-erroryes0:00:05
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 53, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 745, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 959, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 120, in write_logs
    services2id: Dict[ServiceName, ContainerID] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 63, in get_services_id
    raise DockerComposeFail(msg, output=output.decode(), names=names) from e
duckietown_challenges_runner.docker_compose.DockerComposeFail: Cannot get process ids
│ output: ''
│  names: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
58176LFv-simsuccessyes0:32:15
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
58173LFv-simsuccessyes0:17:18
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
58129LFv-simtimeoutyes----No reset possible
52309LFv-simerrorno0:03:47
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140096404600768
- M:video_aido:cmdline(in:/;out:/) 140096404785616
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
52306LFv-simerrorno0:10:42
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140087451376608
- M:video_aido:cmdline(in:/;out:/) 140087451425568
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
52301LFv-simerrorno0:08:57
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140061657565936
- M:video_aido:cmdline(in:/;out:/) 140061657563872
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
52253LFv-simhost-errorno0:03:40
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
52252LFv-simtimeoutno0:08:17
Timeout because eval [...]
Timeout because evaluator contacted us
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
52237LFv-simerrorno0:09:02
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140490429497008
- M:video_aido:cmdline(in:/;out:/) 140490429513200
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
52225LFv-simerrorno0:10:05
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140704501056176
- M:video_aido:cmdline(in:/;out:/) 140704501055888
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
41686LFv-simsuccessno0:09:32
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38137LFv-simerrorno0:00:41
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.

No results found! Something very wrong. 
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.49-py3.8.egg/duckietown_challenges_runner/runner.py", line 1088, in run_one
    cr = read_challenge_results(wd)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/challenge_results.py", line 90, in read_challenge_results
    raise NoResultsFound(msg)
duckietown_challenges.challenge_results.NoResultsFound: File '/media/sdb/duckietown-tmp-evals/production/aido5-LF-sim-validation/submission9361/LFv-sim-mont02-80325a328f54-1-job38137-a-wd/challenge-results/challenge_results.yaml' does not exist.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
36273LFv-simsuccessno0:14:56
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
36272LFv-simsuccessno0:15:54
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
35713LFv-simsuccessno0:01:05
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
35329LFv-simerrorno0:21:57
The result file is n [...]
The result file is not found in working dir /tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9361/LFv-sim-reg02-1b92df2e7e91-1-job35329:

File '/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9361/LFv-sim-reg02-1b92df2e7e91-1-job35329/challenge-results/challenge_results.yaml' does not exist.

This usually means that the evaluator did not finish and some times that there was an import error.
Check the evaluator log to see what happened.

List of all files:

 -/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9361/LFv-sim-reg02-1b92df2e7e91-1-job35329/docker-compose.original.yaml
-/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9361/LFv-sim-reg02-1b92df2e7e91-1-job35329/docker-compose.yaml
-/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9361/LFv-sim-reg02-1b92df2e7e91-1-job35329/logs/challenges-runner/stdout.log
-/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9361/LFv-sim-reg02-1b92df2e7e91-1-job35329/logs/challenges-runner/stderr.log
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
34967LFv-simsuccessno0:22:59
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
34817LFv-simsuccessno0:24:09
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
34778LFv-simabortedno0:00:00
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.31-py3.8.egg/duckietown_challenges_runner/runner.py", line 626, in get_cr
    os.makedirs(wd)
  File "/usr/lib/python3.8/os.py", line 223, in makedirs
    mkdir(name, mode)
OSError: [Errno 28] No space left on device: '/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9361/LFv-sim-reg04-09ab9b94fef9-1-job34778'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
34775LFv-simabortedno0:00:00
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.31-py3.8.egg/duckietown_challenges_runner/runner.py", line 626, in get_cr
    os.makedirs(wd)
  File "/usr/lib/python3.8/os.py", line 223, in makedirs
    mkdir(name, mode)
OSError: [Errno 28] No space left on device: '/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9361/LFv-sim-reg04-09ab9b94fef9-1-job34775'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
34764LFv-simabortedno0:00:00
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.31-py3.8.egg/duckietown_challenges_runner/runner.py", line 626, in get_cr
    os.makedirs(wd)
  File "/usr/lib/python3.8/os.py", line 223, in makedirs
    mkdir(name, mode)
OSError: [Errno 28] No space left on device: '/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9361/LFv-sim-reg09-bc9a115e10b3-1-job34764'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
34756LFv-simabortedno0:24:21
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.31-py3.8.egg/duckietown_challenges_runner/runner.py", line 671, in get_cr
    uploaded = upload_files(wd, aws_config, copy_to_machine_cache=copy_to_machine_cache)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.31-py3.8.egg/duckietown_challenges_runner/runner.py", line 1406, in upload_files
    uploaded = upload(aws_config, toupload, copy_to_machine_cache=copy_to_machine_cache)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.31-py3.8.egg/duckietown_challenges_runner/runner.py", line 1654, in upload
    copy_to_cache(realfile, sha256hex)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.31-py3.8.egg/duckietown_challenges_runner/runner_cache.py", line 42, in copy_to_cache
    shutil.copy(fn, have)
  File "/usr/lib/python3.8/shutil.py", line 415, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/lib/python3.8/shutil.py", line 272, in copyfile
    _fastcopy_sendfile(fsrc, fdst)
  File "/usr/lib/python3.8/shutil.py", line 163, in _fastcopy_sendfile
    raise err from None
  File "/usr/lib/python3.8/shutil.py", line 149, in _fastcopy_sendfile
    sent = os.sendfile(outfd, infd, offset, blocksize)
OSError: [Errno 28] No space left on device: '/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9361/LFv-sim-reg10-9558a63416c8-1-job34756/docker-compose.yaml' -> '/tmp/duckietown/DT18/evaluator/cache/by-value/sha256hex/67faf54fcfe00e1849d35eb1406b9c1974edc5ee217d928db820e35c9972d25a'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
34755LFv-simabortedno0:24:46
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.31-py3.8.egg/duckietown_challenges_runner/runner.py", line 671, in get_cr
    uploaded = upload_files(wd, aws_config, copy_to_machine_cache=copy_to_machine_cache)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.31-py3.8.egg/duckietown_challenges_runner/runner.py", line 1406, in upload_files
    uploaded = upload(aws_config, toupload, copy_to_machine_cache=copy_to_machine_cache)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.31-py3.8.egg/duckietown_challenges_runner/runner.py", line 1654, in upload
    copy_to_cache(realfile, sha256hex)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.31-py3.8.egg/duckietown_challenges_runner/runner_cache.py", line 42, in copy_to_cache
    shutil.copy(fn, have)
  File "/usr/lib/python3.8/shutil.py", line 415, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/lib/python3.8/shutil.py", line 272, in copyfile
    _fastcopy_sendfile(fsrc, fdst)
  File "/usr/lib/python3.8/shutil.py", line 163, in _fastcopy_sendfile
    raise err from None
  File "/usr/lib/python3.8/shutil.py", line 149, in _fastcopy_sendfile
    sent = os.sendfile(outfd, infd, offset, blocksize)
OSError: [Errno 28] No space left on device: '/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9361/LFv-sim-reg05-e09f63c9a256-1-job34755/docker-compose.original.yaml' -> '/tmp/duckietown/DT18/evaluator/cache/by-value/sha256hex/296d780c92941bc56ad77962a73bd5d721da70d85542869beb7b7267b92720b8'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible