Duckietown Challenges Home Challenges Submissions

Submission 13576

Submission13576
Competingyes
Challengeaido-LFVI_multi-sim-validation
UserMárton Tim 🇭🇺
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobssim-0of4: 105567 sim-1of4: 105536 sim-2of4: 105555 sim-3of4: 105557
Next
User label3626
Admin priority50
Blessingn/a
User priority50

105567

Click the images to see detailed statistics about the episode.

udem-000

challenge-solution-output

105536

Click the images to see detailed statistics about the episode.

4way-000

challenge-solution-output

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
105567sim-0of4successyes0:06:44
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median17.30000000000011
in-drivable-lane_median0.9500000000000072
driven_lanedir_consec_median4.595679228515795
deviation-center-line_median1.2031099735550883


other stats
deviation-center-line_max2.5173242687643107
deviation-center-line_mean1.4352429772162971
deviation-center-line_min0.8174276929907005
deviation-heading_max8.488767093733564
deviation-heading_mean4.009814164763104
deviation-heading_median3.275958583175495
deviation-heading_min0.9985723989678624
distance-from-start_max3.1773726759750174
distance-from-start_mean1.9327646315689035
distance-from-start_median1.7512158119565902
distance-from-start_min1.051254226387417
driven_any_max8.029006774970583
driven_any_mean5.107181904828435
driven_any_median5.640159529581741
driven_any_min1.1194017851796758
driven_lanedir_consec_max7.326255309831163
driven_lanedir_consec_mean4.393182461452074
driven_lanedir_consec_min1.0551160789455454
driven_lanedir_max7.347159484846827
driven_lanedir_mean4.523527336196031
driven_lanedir_median4.845870171736838
driven_lanedir_min1.0552095164636184
in-drivable-lane_max7.150000000000087
in-drivable-lane_mean2.2625000000000255
in-drivable-lane_min0.0
per-episodes
details{"udem-000-ego0": {"driven_any": 5.6415363280258966, "survival_time": 17.30000000000011, "driven_lanedir": 5.261195007662264, "in-drivable-lane": 0.25000000000000355, "deviation-heading": 8.488767093733564, "distance-from-start": 3.1773726759750174, "deviation-center-line": 1.5502736328209956, "driven_lanedir_consec": 5.182633467661285}, "udem-000-ego1": {"driven_any": 1.1194017851796758, "survival_time": 17.30000000000011, "driven_lanedir": 1.0552095164636184, "in-drivable-lane": 0.0, "deviation-heading": 0.9985723989678624, "distance-from-start": 1.051254226387417, "deviation-center-line": 2.5173242687643107, "driven_lanedir_consec": 1.0551160789455454}, "udem-000-ego2": {"driven_any": 8.029006774970583, "survival_time": 17.30000000000011, "driven_lanedir": 7.347159484846827, "in-drivable-lane": 1.6500000000000106, "deviation-heading": 3.1939858228122673, "distance-from-start": 1.8084739415920228, "deviation-center-line": 0.8174276929907005, "driven_lanedir_consec": 7.326255309831163}, "udem-000-ego3": {"driven_any": 5.638782731137585, "survival_time": 17.30000000000011, "driven_lanedir": 4.430545335811413, "in-drivable-lane": 7.150000000000087, "deviation-heading": 3.357931343538723, "distance-from-start": 1.6939576823211575, "deviation-center-line": 0.8559463142891816, "driven_lanedir_consec": 4.008724989370305}}
simulation-passed1
survival_time_max17.30000000000011
survival_time_mean17.30000000000011
survival_time_min17.30000000000011
No reset possible
105557sim-3of4successyes0:01:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
skipped1
No reset possible
105555sim-2of4successyes0:01:09
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
skipped1
No reset possible
105551sim-2of4successyes0:01:09
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
105550sim-3of4successyes0:01:06
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
105547sim-0of4successyes0:05:20
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
105543sim-2of4successyes0:01:09
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
105542sim-3of4successyes0:01:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
105541sim-0of4host-erroryes0:01:21
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 298, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 338, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego3" aborted with the following error:

error in ego3 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 322, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 29, in init
              ||     self.model = RLlibModel(0, experiment_idx=0, checkpoint_idx=0, logger=context)
              ||   File "/submission/model.py", line 42, in __init__
              ||     dummy_env = wrap_env(config["env_config"], extra_config={
              ||   File "/submission/duckietown_utils/env.py", line 46, in wrap_env
              ||     env = SegmentObsWrapper(env, model=extra_config['model'])
              ||   File "/submission/duckietown_utils/wrappers/SegmentObsWrapper.py", line 43, in __init__
              ||     self.model.cuda()
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 463, in cuda
              ||     return self._apply(lambda t: t.cuda(device))
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 359, in _apply
              ||     module._apply(fn)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 359, in _apply
              ||     module._apply(fn)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 381, in _apply
              ||     param_applied = fn(param)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 463, in <lambda>
              ||     return self._apply(lambda t: t.cuda(device))
              || RuntimeError: CUDA error: out of memory
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 306, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 298, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 338, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego3" aborted with the following error:

error in ego3 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 322, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 29, in init
              ||     self.model = RLlibModel(0, experiment_idx=0, checkpoint_idx=0, logger=context)
              ||   File "/submission/model.py", line 42, in __init__
              ||     dummy_env = wrap_env(config["env_config"], extra_config={
              ||   File "/submission/duckietown_utils/env.py", line 46, in wrap_env
              ||     env = SegmentObsWrapper(env, model=extra_config['model'])
              ||   File "/submission/duckietown_utils/wrappers/SegmentObsWrapper.py", line 43, in __init__
              ||     self.model.cuda()
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 463, in cuda
              ||     return self._apply(lambda t: t.cuda(device))
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 359, in _apply
              ||     module._apply(fn)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 359, in _apply
              ||     module._apply(fn)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 381, in _apply
              ||     param_applied = fn(param)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 463, in <lambda>
              ||     return self._apply(lambda t: t.cuda(device))
              || RuntimeError: CUDA error: out of memory
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
105540sim-3of4successyes0:01:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
105539sim-2of4successyes0:01:09
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
105536sim-1of4successyes0:03:17
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median4.749999999999991
in-drivable-lane_median0.37499999999999867
driven_lanedir_consec_median1.3461137381912445
deviation-center-line_median0.3100673308427664


other stats
deviation-center-line_max0.36581936795974845
deviation-center-line_mean0.31013941571096204
deviation-center-line_min0.2546036331985669
deviation-heading_max2.217872295390038
deviation-heading_mean1.4285781618671891
deviation-heading_median1.3977234883105054
deviation-heading_min0.700993375457709
distance-from-start_max1.633203569285496
distance-from-start_mean1.3357370528540304
distance-from-start_median1.35432001222902
distance-from-start_min1.0011046176725855
driven_any_max1.8717790822321103
driven_any_mean1.777766659112212
driven_any_median1.7995605112294657
driven_any_min1.6401665317578062
driven_lanedir_consec_max1.807055301393011
driven_lanedir_consec_mean1.3451175071332344
driven_lanedir_consec_min0.8811872507574376
driven_lanedir_max1.8507604770965893
driven_lanedir_mean1.430103991090534
driven_lanedir_median1.4821209349845224
driven_lanedir_min0.9054136172965016
in-drivable-lane_max1.2999999999999956
in-drivable-lane_mean0.5124999999999982
in-drivable-lane_min0.0
per-episodes
details{"4way-000-ego0": {"driven_any": 1.841767784048266, "survival_time": 4.749999999999991, "driven_lanedir": 1.8070553214161336, "in-drivable-lane": 0.0, "deviation-heading": 0.9342163633862156, "distance-from-start": 1.3036912861820276, "deviation-center-line": 0.2546036331985669, "driven_lanedir_consec": 1.807055301393011}, "4way-000-ego1": {"driven_any": 1.7573532384106654, "survival_time": 4.749999999999991, "driven_lanedir": 1.1571865485529107, "in-drivable-lane": 0.7499999999999973, "deviation-heading": 2.217872295390038, "distance-from-start": 1.633203569285496, "deviation-center-line": 0.36581936795974845, "driven_lanedir_consec": 1.1421372937613687}, "4way-000-ego2": {"driven_any": 1.8717790822321103, "survival_time": 4.749999999999991, "driven_lanedir": 1.8507604770965893, "in-drivable-lane": 0.0, "deviation-heading": 0.700993375457709, "distance-from-start": 1.4049487382760126, "deviation-center-line": 0.29416794940876584, "driven_lanedir_consec": 1.55009018262112}, "4way-000-ego3": {"driven_any": 1.6401665317578062, "survival_time": 4.749999999999991, "driven_lanedir": 0.9054136172965016, "in-drivable-lane": 1.2999999999999956, "deviation-heading": 1.861230613234795, "distance-from-start": 1.0011046176725855, "deviation-center-line": 0.3259667122767669, "driven_lanedir_consec": 0.8811872507574376}}
simulation-passed1
survival_time_max4.749999999999991
survival_time_mean4.749999999999991
survival_time_min4.749999999999991
No reset possible
105524sim-1of4successyes0:05:26
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100675sim-2of4successno0:01:09
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100674sim-3of4successno0:01:06
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100672sim-0of4failedno0:00:47
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100667sim-1of4failedno0:00:59
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100664sim-3of4successno0:01:07
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100662sim-0of4failedno0:00:59
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100659sim-2of4successno0:01:09
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100658sim-1of4failedno0:00:59
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100655sim-2of4successno0:01:10
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100654sim-3of4successno0:01:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100653sim-0of4failedno0:01:02
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100650sim-1of4failedno0:00:56
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100648sim-0of4failedno0:00:56
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100647sim-3of4successno0:01:07
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100646sim-2of4successno0:01:05
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
91561sim-0of4successno0:13:32
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
91558sim-2of4successno0:01:49
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
91554sim-2of4successno0:01:57
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
91553sim-2of4successno0:01:58
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
91542sim-1of4successno0:11:36
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
91539sim-3of4successno0:01:53
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
91537sim-3of4successno0:01:48
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
91536sim-3of4successno0:01:49
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
74829sim-2of4successno0:00:57
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
74824sim-0of4successno0:07:00
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
74823sim-2of4successno0:00:59
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
74808sim-2of4successno0:00:56
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
74803sim-0of4failedno0:01:25
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 601, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 135, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 218, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 304, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 331, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "ego1" aborted:

error in ego1 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 355, in loop
              ||     handle_message_node(parsed, receiver0, context0)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 531, in handle_message_node
              ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 90, in on_received_get_commands
              ||     pwm_left, pwm_right = self.compute_action(self.current_image)
              ||   File "solution.py", line 84, in compute_action
              ||     action = self.model.predict(observation)
              ||   File "/submission/model.py", line 71, in predict
              ||     observation = wrapper.observation(observation)
              ||   File "/submission/duckietown_utils/wrappers/SegmentObsWrapper.py", line 56, in observation
              ||     _, pred = torch.max(self.model.forward(obs_prep), 1)
              ||   File "/submission/duckietown_utils/wrappers/my_models/tiramisu.py", line 145, in forward
              ||     x = self.featureExtractor(x)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 727, in _call_impl
              ||     result = self.forward(*input, **kwargs)
              ||   File "/submission/duckietown_utils/wrappers/my_models/tiramisu.py", line 96, in forward
              ||     out = self.transDownBlocks[i](out)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 727, in _call_impl
              ||     result = self.forward(*input, **kwargs)
              ||   File "/submission/duckietown_utils/wrappers/my_models/layers.py", line 55, in forward
              ||     return super().forward(x)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/container.py", line 117, in forward
              ||     input = module(input)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 727, in _call_impl
              ||     result = self.forward(*input, **kwargs)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/conv.py", line 423, in forward
              ||     return self._conv_forward(input, self.weight)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/conv.py", line 419, in _conv_forward
              ||     return F.conv2d(input, weight, self.bias, self.stride,
              || RuntimeError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 14.76 GiB total capacity; 100.10 MiB already allocated; 31.44 MiB free; 118.00 MiB reserved in total by PyTorch)
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 315, in main
    length_s = await run_episode(
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 605, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
74788sim-3of4successno0:00:59
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
74781sim-1of4successno0:05:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
74779sim-3of4successno0:01:02
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
74777sim-3of4successno0:00:56
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
62092365failedno0:04:36
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 190, in main
    raise InvalidSubmission(msg)
duckietown_challenges.exceptions.InvalidSubmission: Timeout during connection to ego0: <SignalTimeout in state: 2>
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
62091365host-errorno0:02:54
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 322, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 29, in init
              ||     self.model = RLlibModel(0, experiment_idx=0, checkpoint_idx=0, logger=context)
              ||   File "/submission/model.py", line 42, in __init__
              ||     dummy_env = wrap_env(config["env_config"], extra_config={
              ||   File "/submission/duckietown_utils/env.py", line 46, in wrap_env
              ||     env = SegmentObsWrapper(env, model=extra_config['model'])
              ||   File "/submission/duckietown_utils/wrappers/SegmentObsWrapper.py", line 43, in __init__
              ||     self.model.cuda()
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 463, in cuda
              ||     return self._apply(lambda t: t.cuda(device))
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 359, in _apply
              ||     module._apply(fn)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 359, in _apply
              ||     module._apply(fn)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 381, in _apply
              ||     param_applied = fn(param)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 463, in <lambda>
              ||     return self._apply(lambda t: t.cuda(device))
              || RuntimeError: CUDA error: out of memory
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 322, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 29, in init
              ||     self.model = RLlibModel(0, experiment_idx=0, checkpoint_idx=0, logger=context)
              ||   File "/submission/model.py", line 42, in __init__
              ||     dummy_env = wrap_env(config["env_config"], extra_config={
              ||   File "/submission/duckietown_utils/env.py", line 46, in wrap_env
              ||     env = SegmentObsWrapper(env, model=extra_config['model'])
              ||   File "/submission/duckietown_utils/wrappers/SegmentObsWrapper.py", line 43, in __init__
              ||     self.model.cuda()
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 463, in cuda
              ||     return self._apply(lambda t: t.cuda(device))
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 359, in _apply
              ||     module._apply(fn)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 359, in _apply
              ||     module._apply(fn)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 381, in _apply
              ||     param_applied = fn(param)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 463, in <lambda>
              ||     return self._apply(lambda t: t.cuda(device))
              || RuntimeError: CUDA error: out of memory
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible