Submission 9243

Submission

9243

Competing

yes

Challenge

aido5-LF-sim-validation

User

Melisande Teng

Date submitted

2020-10-23 21:29:22+00:00

Last status update

2021-12-04 20:52:32+00:00

Complete

complete

Details

Evaluation is complete.

Sisters

Result

💚

Jobs

LFv-sim: 58471

User label

exercise_ros_template

Admin priority

Blessing

n/a

User priority

58471

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs

Job ID

step

status

up to date

date started

date completed

duration

message

58471

LFv-sim

success

yes

2020-12-03 23:44:01+00:00

2020-12-03 23:52:27+00:00

0:08:26

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

driven_lanedir_consec_median	0.6946411742740501
survival_time_median	8.949999999999992
deviation-center-line_median	0.2490145281345859
in-drivable-lane_median	4.725000000000035

other stats

agent_compute-ego0_max	0.03625090028986584
agent_compute-ego0_mean	0.026304224826030943
agent_compute-ego0_median	0.02729881665983253
agent_compute-ego0_min	0.014368365694592882
complete-iteration_max	0.23043077618050176
complete-iteration_mean	0.20331116213856257
complete-iteration_median	0.2040423786318229
complete-iteration_min	0.1747291151101027
deviation-center-line_max	1.2692857390580914
deviation-center-line_mean	0.48429665690615753
deviation-center-line_min	0.16987183229736666
deviation-heading_max	2.805421957517778
deviation-heading_mean	1.5469753402969382
deviation-heading_median	1.4008306798333383
deviation-heading_min	0.5808180440032985
driven_any_max	4.928671275017078
driven_any_mean	2.376278391277715
driven_any_median	1.6582508635711823
driven_any_min	1.259940562951416
driven_lanedir_consec_max	3.954751948934782
driven_lanedir_consec_mean	1.5042740607409275
driven_lanedir_consec_min	0.673061945480828
driven_lanedir_max	3.954751948934782
driven_lanedir_mean	1.5113147508306135
driven_lanedir_median	0.7033969510816256
driven_lanedir_min	0.6837131522244205
get_duckie_state_max	1.6372520606834573e-06
get_duckie_state_mean	1.5101590577208847e-06
get_duckie_state_median	1.5170711372888502e-06
get_duckie_state_min	1.3692418956223814e-06
get_robot_state_max	0.004000513703673036
get_robot_state_mean	0.00385998955200634
get_robot_state_median	0.003874929859509382
get_robot_state_min	0.0036895847853335583
get_state_dump_max	0.0051451446293117285
get_state_dump_mean	0.005013499408202094
get_state_dump_median	0.005065172650992578
get_state_dump_min	0.00477850770151149
get_ui_image_max	0.03488226443029649
get_ui_image_mean	0.03125658611549183
get_ui_image_median	0.03110982782736762
get_ui_image_min	0.02792442437693559
in-drivable-lane_max	5.199999999999998
in-drivable-lane_mean	4.500000000000014
in-drivable-lane_min	3.349999999999988
per-episodes	details {"LF-norm-loop-000-ego0": {"driven_any": 4.928671275017078, "get_ui_image": 0.028202656323347633, "step_physics": 0.10001232711280264, "survival_time": 24.550000000000217, "driven_lanedir": 3.954751948934782, "get_state_dump": 0.005011474698539672, "get_robot_state": 0.003788884093121784, "sim_render-ego0": 0.003906927942260494, "get_duckie_state": 1.520645327684356e-06, "in-drivable-lane": 5.000000000000071, "deviation-heading": 2.805421957517778, "agent_compute-ego0": 0.01884794235229492, "complete-iteration": 0.1747291151101027, "set_robot_commands": 0.00236691255879596, "deviation-center-line": 1.2692857390580914, "driven_lanedir_consec": 3.954751948934782, "sim_compute_sim_state": 0.010370805980713388, "sim_compute_performance-ego0": 0.002116033216802085}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.6477547497971692, "get_ui_image": 0.03488226443029649, "step_physics": 0.13236257483839323, "survival_time": 8.899999999999991, "driven_lanedir": 0.7012247058395715, "get_state_dump": 0.00477850770151149, "get_robot_state": 0.0036895847853335583, "sim_render-ego0": 0.003850958200806346, "get_duckie_state": 1.3692418956223814e-06, "in-drivable-lane": 4.449999999999999, "deviation-heading": 2.1726931066388384, "agent_compute-ego0": 0.03625090028986584, "complete-iteration": 0.23043077618050176, "set_robot_commands": 0.002399191510077962, "deviation-center-line": 0.32456372962930324, "driven_lanedir_consec": 0.673061945480828, "sim_compute_sim_state": 0.010060651342296069, "sim_compute_performance-ego0": 0.00205314225990679}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.259940562951416, "get_ui_image": 0.034016999331387604, "step_physics": 0.13819114478318006, "survival_time": 7.099999999999983, "driven_lanedir": 0.6837131522244205, "get_state_dump": 0.0051451446293117285, "get_robot_state": 0.004000513703673036, "sim_render-ego0": 0.004088335103921957, "get_duckie_state": 1.6372520606834573e-06, "in-drivable-lane": 3.349999999999988, "deviation-heading": 0.6289682530278384, "agent_compute-ego0": 0.014368365694592882, "complete-iteration": 0.21361932554445068, "set_robot_commands": 0.0024550044453227437, "deviation-center-line": 0.16987183229736666, "driven_lanedir_consec": 0.6837131522244205, "sim_compute_sim_state": 0.009037046165733071, "sim_compute_performance-ego0": 0.002205430210887135}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.6687469773451955, "get_ui_image": 0.02792442437693559, "step_physics": 0.10713295778516908, "survival_time": 8.999999999999993, "driven_lanedir": 0.7055691963236796, "get_state_dump": 0.005118870603445485, "get_robot_state": 0.00396097562589698, "sim_render-ego0": 0.004088333298488216, "get_duckie_state": 1.5134969468933442e-06, "in-drivable-lane": 5.199999999999998, "deviation-heading": 0.5808180440032985, "agent_compute-ego0": 0.035749690967370136, "complete-iteration": 0.19446543171919511, "set_robot_commands": 0.0024597131086317872, "deviation-center-line": 0.17346532663986858, "driven_lanedir_consec": 0.7055691963236796, "sim_compute_sim_state": 0.005773265058823054, "sim_compute_performance-ego0": 0.002153334696648529}}
set_robot_commands_max	0.0024597131086317872
set_robot_commands_mean	0.002420205405707113
set_robot_commands_median	0.002427097977700353
set_robot_commands_min	0.00236691255879596
sim_compute_performance-ego0_max	0.002205430210887135
sim_compute_performance-ego0_mean	0.0021319850960611346
sim_compute_performance-ego0_median	0.002134683956725307
sim_compute_performance-ego0_min	0.00205314225990679
sim_compute_sim_state_max	0.010370805980713388
sim_compute_sim_state_mean	0.008810442136891396
sim_compute_sim_state_median	0.009548848754014568
sim_compute_sim_state_min	0.005773265058823054
sim_render-ego0_max	0.004088335103921957
sim_render-ego0_mean	0.003983638636369254
sim_render-ego0_median	0.003997630620374355
sim_render-ego0_min	0.003850958200806346
simulation-passed	1
step_physics_max	0.13819114478318006
step_physics_mean	0.11942475112988624
step_physics_median	0.11974776631178116
step_physics_min	0.10001232711280264
survival_time_max	24.550000000000217
survival_time_mean	12.387500000000044
survival_time_min	7.099999999999983

No reset possible

58470

LFv-sim

success

yes

2020-12-03 23:42:43+00:00

2020-12-03 23:58:50+00:00

0:16:07

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

58469

LFv-sim

success

yes

2020-12-03 23:42:13+00:00

2020-12-03 23:49:32+00:00

0:07:19

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

58466

LFv-sim

success

yes

2020-12-03 23:40:31+00:00

2020-12-03 23:47:24+00:00

0:06:53

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

52418

LFv-sim

error

2020-12-02 11:36:46+00:00

2020-12-02 11:39:07+00:00

0:02:21

InvalidEvaluator: Tr [...]

InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140447939359312
- M:video_aido:cmdline(in:/;out:/) 140447939359024
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

52416

LFv-sim

error

2020-12-02 11:36:18+00:00

2020-12-02 11:41:54+00:00

0:05:36

InvalidEvaluator: Tr [...]

InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139988008809328
- M:video_aido:cmdline(in:/;out:/) 139988008809232
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

52414

LFv-sim

error

2020-12-02 11:36:03+00:00

2020-12-02 11:39:25+00:00

0:03:22

InvalidEvaluator: Tr [...]

InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140057950611584
- M:video_aido:cmdline(in:/;out:/) 140057950438976
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

52404

LFv-sim

error

2020-12-02 11:34:05+00:00

2020-12-02 11:37:26+00:00

0:03:21

InvalidEvaluator: Tr [...]

InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140057442512080
- M:video_aido:cmdline(in:/;out:/) 140057442509824
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

52399

LFv-sim

error

2020-12-02 11:33:42+00:00

2020-12-02 11:37:40+00:00

0:03:58

InvalidEvaluator: Tr [...]

InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140199433393344
- M:video_aido:cmdline(in:/;out:/) 140199435109760
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

41773

LFv-sim

success

2020-11-08 14:15:43+00:00

2020-11-08 14:23:23+00:00

0:07:40

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

38310

LFv-sim

success

2020-11-04 01:56:49+00:00

2020-11-04 02:04:18+00:00

0:07:29

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

36406

LFv-sim

error

2020-11-02 05:14:12+00:00

2020-11-02 05:14:54+00:00

0:00:42

The container "solut [...]

The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.

No results found! Something very wrong. 
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1088, in run_one
    cr = read_challenge_results(wd)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/challenge_results.py", line 90, in read_challenge_results
    raise NoResultsFound(msg)
duckietown_challenges.challenge_results.NoResultsFound: File '/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9243/LFv-sim-Sandy2-sandy-1-job36406-a-wd/challenge-results/challenge_results.yaml' does not exist.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

36405

LFv-sim

error

2020-11-02 05:13:34+00:00

2020-11-02 05:14:16+00:00

0:00:42

The container "solut [...]

The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.

No results found! Something very wrong. 
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1088, in run_one
    cr = read_challenge_results(wd)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/challenge_results.py", line 90, in read_challenge_results
    raise NoResultsFound(msg)
duckietown_challenges.challenge_results.NoResultsFound: File '/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9243/LFv-sim-Sandy1-sandy-1-job36405-a-wd/challenge-results/challenge_results.yaml' does not exist.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

35834

LFv-sim

error

2020-11-01 04:24:58+00:00

2020-11-01 04:25:45+00:00

0:00:47

The container "solut [...]

The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.

No results found! Something very wrong. 
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1063, in run_one
    cr = read_challenge_results(wd)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/challenge_results.py", line 90, in read_challenge_results
    raise NoResultsFound(msg)
duckietown_challenges.challenge_results.NoResultsFound: File '/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9243/LFv-sim-noname-sandy-1-job35834-a-wd/challenge-results/challenge_results.yaml' does not exist.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

35833

LFv-sim

success

2020-11-01 04:24:31+00:00

2020-11-01 04:25:34+00:00

0:01:03

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

35417

LFv-sim

error

2020-10-31 05:43:28+00:00

2020-10-31 06:04:52+00:00

0:21:24

The result file is n [...]

The result file is not found in working dir /tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9243/LFv-sim-reg05-b2dee9d94ee0-1-job35417:

File '/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9243/LFv-sim-reg05-b2dee9d94ee0-1-job35417/challenge-results/challenge_results.yaml' does not exist.

This usually means that the evaluator did not finish and some times that there was an import error.
Check the evaluator log to see what happened.

List of all files:

 -/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9243/LFv-sim-reg05-b2dee9d94ee0-1-job35417/docker-compose.original.yaml
-/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9243/LFv-sim-reg05-b2dee9d94ee0-1-job35417/docker-compose.yaml
-/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9243/LFv-sim-reg05-b2dee9d94ee0-1-job35417/logs/challenges-runner/stdout.log
-/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9243/LFv-sim-reg05-b2dee9d94ee0-1-job35417/logs/challenges-runner/stderr.log

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

35062

LFv-sim

success

2020-10-29 06:57:22+00:00

2020-10-29 07:19:51+00:00

0:22:29

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

34526

LFv-sim

success

2020-10-23 21:31:19+00:00

2020-10-23 21:57:33+00:00

0:26:14

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

34525

LFv-sim

success

2020-10-23 21:30:50+00:00

2020-10-23 21:57:50+00:00

0:27:00

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible