Submission 9333

Submission

9333

Competing

yes

Challenge

aido5-LF-sim-validation

User

Himanshu Arora 🇨🇦

Date submitted

2020-10-25 23:50:30+00:00

Last status update

2021-12-04 20:52:33+00:00

Complete

complete

Details

Evaluation is complete.

Sisters

Result

💚

Jobs

LFv-sim: 58165

User label

exercise_ros_template

Admin priority

Blessing

n/a

User priority

58165

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs

Job ID

step

status

up to date

date started

date completed

duration

message

58165

LFv-sim

success

yes

2020-12-03 21:57:32+00:00

2020-12-03 22:03:44+00:00

0:06:12

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

driven_lanedir_consec_median	0.48892942179856824
survival_time_median	5.249999999999989
deviation-center-line_median	0.13191648840869452
in-drivable-lane_median	3.2499999999999902

other stats

agent_compute-ego0_max	0.013665390014648438
agent_compute-ego0_mean	0.013208485049933596
agent_compute-ego0_median	0.013289727090479254
agent_compute-ego0_min	0.012589096004127436
complete-iteration_max	0.21291047146445827
complete-iteration_mean	0.1858831868237345
complete-iteration_median	0.18622185906647765
complete-iteration_min	0.15817855769752437
deviation-center-line_max	0.1643959468172477
deviation-center-line_mean	0.12557847619610993
deviation-center-line_min	0.07408498114980296
deviation-heading_max	1.3085060007319516
deviation-heading_mean	0.8127054713150081
deviation-heading_median	0.7946842392500675
deviation-heading_min	0.35294740602794605
driven_any_max	3.0258559942245338
driven_any_mean	1.7592592311392878
driven_any_median	1.470999276751933
driven_any_min	1.0691823768287514
driven_lanedir_consec_max	0.6494315759471684
driven_lanedir_consec_mean	0.4751876951392385
driven_lanedir_consec_min	0.27346036101264914
driven_lanedir_max	0.6494315759471684
driven_lanedir_mean	0.4751876951392385
driven_lanedir_median	0.48892942179856824
driven_lanedir_min	0.27346036101264914
get_duckie_state_max	1.4505888286389803e-06
get_duckie_state_mean	1.3944480024016262e-06
get_duckie_state_median	1.3795673337757078e-06
get_duckie_state_min	1.3680685134161086e-06
get_robot_state_max	0.003956445625850132
get_robot_state_mean	0.003788342308924565
get_robot_state_median	0.0037807035381816497
get_robot_state_min	0.003635516533484826
get_state_dump_max	0.00509497080093775
get_state_dump_mean	0.004867310368204996
get_state_dump_median	0.00486911273839181
get_state_dump_min	0.004636045195098616
get_ui_image_max	0.03739292496129086
get_ui_image_mean	0.03178724224019208
get_ui_image_median	0.031230635848237483
get_ui_image_min	0.02729477230300251
in-drivable-lane_max	8.100000000000005
in-drivable-lane_mean	4.099999999999995
in-drivable-lane_min	1.7999999999999936
per-episodes	details {"LF-norm-loop-000-ego0": {"driven_any": 3.0258559942245338, "get_ui_image": 0.029581252122536685, "step_physics": 0.0999961828574156, "survival_time": 9.700000000000005, "driven_lanedir": 0.27346036101264914, "get_state_dump": 0.00509497080093775, "get_robot_state": 0.0037706570747571113, "sim_render-ego0": 0.0038355546119885568, "get_duckie_state": 1.3816050994090544e-06, "in-drivable-lane": 8.100000000000005, "deviation-heading": 1.175100072434885, "agent_compute-ego0": 0.013122786008394682, "complete-iteration": 0.17008168880756086, "set_robot_commands": 0.0022696177164713543, "deviation-center-line": 0.1643959468172477, "driven_lanedir_consec": 0.27346036101264914, "sim_compute_sim_state": 0.010300144782433143, "sim_compute_performance-ego0": 0.002016911139855018}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.278825974691371, "get_ui_image": 0.03739292496129086, "step_physics": 0.13451550634283768, "survival_time": 4.699999999999991, "driven_lanedir": 0.3629620387619259, "get_state_dump": 0.004893182453356291, "get_robot_state": 0.003790750001606188, "sim_render-ego0": 0.003925915768271998, "get_duckie_state": 1.4505888286389803e-06, "in-drivable-lane": 2.8499999999999934, "deviation-heading": 1.3085060007319516, "agent_compute-ego0": 0.013665390014648438, "complete-iteration": 0.21291047146445827, "set_robot_commands": 0.0022541146529348275, "deviation-center-line": 0.1429278448490799, "driven_lanedir_consec": 0.3629620387619259, "sim_compute_sim_state": 0.010339656629060444, "sim_compute_performance-ego0": 0.002039889285438939}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.0691823768287514, "get_ui_image": 0.03288001957393828, "step_physics": 0.13065356867653982, "survival_time": 4.149999999999993, "driven_lanedir": 0.6494315759471684, "get_state_dump": 0.004845043023427327, "get_robot_state": 0.003956445625850132, "sim_render-ego0": 0.003899815536680675, "get_duckie_state": 1.3680685134161086e-06, "in-drivable-lane": 1.7999999999999936, "deviation-heading": 0.41426840606525006, "agent_compute-ego0": 0.013456668172563826, "complete-iteration": 0.2023620293253944, "set_robot_commands": 0.002301201933906192, "deviation-center-line": 0.07408498114980296, "driven_lanedir_consec": 0.6494315759471684, "sim_compute_sim_state": 0.008161737805321104, "sim_compute_performance-ego0": 0.0021145854677472797}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.6631725788124958, "get_ui_image": 0.02729477230300251, "step_physics": 0.09716660956032255, "survival_time": 5.799999999999987, "driven_lanedir": 0.6148968048352106, "get_state_dump": 0.004636045195098616, "get_robot_state": 0.003635516533484826, "sim_render-ego0": 0.003706972823183761, "get_duckie_state": 1.377529568142361e-06, "in-drivable-lane": 3.649999999999987, "deviation-heading": 0.35294740602794605, "agent_compute-ego0": 0.012589096004127436, "complete-iteration": 0.15817855769752437, "set_robot_commands": 0.0021855606992020565, "deviation-center-line": 0.12090513196830915, "driven_lanedir_consec": 0.6148968048352106, "sim_compute_sim_state": 0.004936483171251085, "sim_compute_performance-ego0": 0.001939154078817775}}
set_robot_commands_max	0.002301201933906192
set_robot_commands_mean	0.0022526237506286075
set_robot_commands_median	0.002261866184703091
set_robot_commands_min	0.0021855606992020565
sim_compute_performance-ego0_max	0.0021145854677472797
sim_compute_performance-ego0_mean	0.002027634992964753
sim_compute_performance-ego0_median	0.0020284002126469787
sim_compute_performance-ego0_min	0.001939154078817775
sim_compute_sim_state_max	0.010339656629060444
sim_compute_sim_state_mean	0.008434505597016445
sim_compute_sim_state_median	0.009230941293877123
sim_compute_sim_state_min	0.004936483171251085
sim_render-ego0_max	0.003925915768271998
sim_render-ego0_mean	0.003842064685031248
sim_render-ego0_median	0.003867685074334616
sim_render-ego0_min	0.003706972823183761
simulation-passed	1
step_physics_max	0.13451550634283768
step_physics_mean	0.11558296685927892
step_physics_median	0.11532487576697772
step_physics_min	0.09716660956032255
survival_time_max	9.700000000000005
survival_time_mean	6.087499999999993
survival_time_min	4.149999999999993

No reset possible

52299

LFv-sim

error

2020-12-02 11:20:30+00:00

2020-12-02 11:24:01+00:00

0:03:31

InvalidEvaluator: Tr [...]

InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140066692280912
- M:video_aido:cmdline(in:/;out:/) 140071669601856
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

52294

LFv-sim

error

2020-12-02 11:19:32+00:00

2020-12-02 11:21:01+00:00

0:01:29

InvalidEvaluator: Tr [...]

InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139794334628976
- M:video_aido:cmdline(in:/;out:/) 139794334626240
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

52289

LFv-sim

error

2020-12-02 11:18:45+00:00

2020-12-02 11:23:22+00:00

0:04:37

InvalidEvaluator: Tr [...]

InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140401020068624
- M:video_aido:cmdline(in:/;out:/) 140405945223488
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

52277

LFv-sim

error

2020-12-02 11:17:30+00:00

2020-12-02 11:22:49+00:00

0:05:19

InvalidEvaluator: Tr [...]

InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139864569476720
- M:video_aido:cmdline(in:/;out:/) 139869472702032
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

52276

LFv-sim

error

2020-12-02 11:17:22+00:00

2020-12-02 11:20:40+00:00

0:03:18

InvalidEvaluator: Tr [...]

InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139701016434336
- M:video_aido:cmdline(in:/;out:/) 139701016477808
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

52271

LFv-sim

error

2020-12-02 11:16:37+00:00

2020-12-02 11:20:31+00:00

0:03:54

InvalidEvaluator: Tr [...]

InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140029394114592
- M:video_aido:cmdline(in:/;out:/) 140032552467040
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

41708

LFv-sim

success

2020-11-08 12:40:06+00:00

2020-11-08 12:44:46+00:00

0:04:40

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

41706

LFv-sim

success

2020-11-08 12:38:57+00:00

2020-11-08 12:43:38+00:00

0:04:41

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

38153

LFv-sim

error

2020-11-04 00:32:15+00:00

2020-11-04 00:32:53+00:00

0:00:38

The container "evalu [...]

The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.

No results found! Something very wrong. 
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.49-py3.8.egg/duckietown_challenges_runner/runner.py", line 1088, in run_one
    cr = read_challenge_results(wd)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/challenge_results.py", line 90, in read_challenge_results
    raise NoResultsFound(msg)
duckietown_challenges.challenge_results.NoResultsFound: File '/media/sdb/duckietown-tmp-evals/production/aido5-LF-sim-validation/submission9333/LFv-sim-mont02-80325a328f54-1-job38153-a-wd/challenge-results/challenge_results.yaml' does not exist.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

38148

LFv-sim

error

2020-11-04 00:31:08+00:00

2020-11-04 00:31:49+00:00

0:00:41

The container "evalu [...]

The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.

No results found! Something very wrong. 
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.49-py3.8.egg/duckietown_challenges_runner/runner.py", line 1088, in run_one
    cr = read_challenge_results(wd)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/challenge_results.py", line 90, in read_challenge_results
    raise NoResultsFound(msg)
duckietown_challenges.challenge_results.NoResultsFound: File '/media/sdb/duckietown-tmp-evals/production/aido5-LF-sim-validation/submission9333/LFv-sim-mont05-227ea22a5fff-1-job38148-a-wd/challenge-results/challenge_results.yaml' does not exist.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

36303

LFv-sim

success

2020-11-02 04:18:49+00:00

2020-11-02 04:25:23+00:00

0:06:34

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

36301

LFv-sim

success

2020-11-02 04:17:33+00:00

2020-11-02 04:24:25+00:00

0:06:52

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

35740

LFv-sim

success

2020-11-01 04:03:13+00:00

2020-11-01 04:04:16+00:00

0:01:03

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

35737

LFv-sim

success

2020-11-01 04:02:39+00:00

2020-11-01 04:03:40+00:00

0:01:01

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

35348

LFv-sim

error

2020-10-31 01:57:29+00:00

2020-10-31 02:06:42+00:00

0:09:13

The result file is n [...]

The result file is not found in working dir /tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9333/LFv-sim-reg01-94a6fab21ac9-1-job35348:

File '/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9333/LFv-sim-reg01-94a6fab21ac9-1-job35348/challenge-results/challenge_results.yaml' does not exist.

This usually means that the evaluator did not finish and some times that there was an import error.
Check the evaluator log to see what happened.

List of all files:

 -/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9333/LFv-sim-reg01-94a6fab21ac9-1-job35348/docker-compose.original.yaml
-/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9333/LFv-sim-reg01-94a6fab21ac9-1-job35348/docker-compose.yaml
-/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9333/LFv-sim-reg01-94a6fab21ac9-1-job35348/logs/challenges-runner/stdout.log
-/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9333/LFv-sim-reg01-94a6fab21ac9-1-job35348/logs/challenges-runner/stderr.log

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

34983

LFv-sim

success

2020-10-29 02:44:36+00:00

2020-10-29 02:54:56+00:00

0:10:20

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

34698

LFv-sim

success

2020-10-25 23:52:42+00:00

2020-10-26 00:03:43+00:00

0:11:01

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

34697

LFv-sim

success

2020-10-25 23:51:56+00:00

2020-10-26 00:02:54+00:00

0:10:58

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

34696

LFv-sim

success

2020-10-25 23:51:50+00:00

2020-10-26 00:19:21+00:00

0:27:31

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible