Duckietown Challenges Home Challenges Submissions

Submission 11555

Submission11555
Competingyes
Challengeaido5-LF-sim-validation
UserYishu Malhotra 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 54384
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

54384

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

See previous jobs for previous versions of challenges
Job IDstepstatusup to datedate starteddate completeddurationmessage
54384LFv-simsuccessyes0:20:23
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.9809531250952004
survival_time_median29.400000000000254
deviation-center-line_median0.8720462615221556
in-drivable-lane_median19.97500000000022


other stats
agent_compute-ego0_max0.013414224867004058
agent_compute-ego0_mean0.01325023013250384
agent_compute-ego0_median0.013276512552026417
agent_compute-ego0_min0.013033670558958474
complete-iteration_max0.29783398933832156
complete-iteration_mean0.24963185017437317
complete-iteration_median0.2448138640606768
complete-iteration_min0.21106568323781744
deviation-center-line_max1.2462828584510095
deviation-center-line_mean0.8291607284825531
deviation-center-line_min0.32626753243489137
deviation-heading_max8.724287359625125
deviation-heading_mean5.218471712435031
deviation-heading_median4.742154857458061
deviation-heading_min2.6652897751988753
driven_any_max7.919935175412725
driven_any_mean4.124919169449713
driven_any_median3.7615348092169
driven_any_min1.056671883952328
driven_lanedir_consec_max2.120572199003452
driven_lanedir_consec_mean1.135560719886501
driven_lanedir_consec_min0.45976443035215064
driven_lanedir_max2.1835728363308498
driven_lanedir_mean1.1513108792183502
driven_lanedir_median0.9809531250952004
driven_lanedir_min0.45976443035215064
get_duckie_state_max1.3554293806381646e-06
get_duckie_state_mean1.3217278127265248e-06
get_duckie_state_median1.3161942317032323e-06
get_duckie_state_min1.2990934068614696e-06
get_robot_state_max0.003891033494998573
get_robot_state_mean0.0037896096139179184
get_robot_state_median0.003800401916166008
get_robot_state_min0.003666601128341085
get_state_dump_max0.004901398611332172
get_state_dump_mean0.004798196486325791
get_state_dump_median0.004784286917853225
get_state_dump_min0.00472281349826454
get_ui_image_max0.036297202768905384
get_ui_image_mean0.0315147394578842
get_ui_image_median0.031141917975098212
get_ui_image_min0.027477919112434992
in-drivable-lane_max40.39999999999905
in-drivable-lane_mean21.112499999999873
in-drivable-lane_min4.100000000000002
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 4.2163774082018834, "get_ui_image": 0.028971883622841567, "step_physics": 0.15146970821479322, "survival_time": 32.800000000000274, "driven_lanedir": 1.152169199661322, "get_state_dump": 0.004758333688094373, "get_robot_state": 0.0037467853482273977, "sim_render-ego0": 0.003797642172199406, "get_duckie_state": 1.3071289526998906e-06, "in-drivable-lane": 22.35000000000026, "deviation-heading": 4.575906682279467, "agent_compute-ego0": 0.013033670558958474, "complete-iteration": 0.2187690426406977, "set_robot_commands": 0.0022126608606161413, "deviation-center-line": 0.9144661167747848, "driven_lanedir_consec": 1.152169199661322, "sim_compute_sim_state": 0.008706462982038385, "sim_compute_performance-ego0": 0.0019890467325846353}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.056671883952328, "get_ui_image": 0.036297202768905384, "step_physics": 0.22024835012235697, "survival_time": 8.999999999999993, "driven_lanedir": 0.45976443035215064, "get_state_dump": 0.004901398611332172, "get_robot_state": 0.003666601128341085, "sim_render-ego0": 0.0037954722978792137, "get_duckie_state": 1.3554293806381646e-06, "in-drivable-lane": 4.100000000000002, "deviation-heading": 2.6652897751988753, "agent_compute-ego0": 0.013414224867004058, "complete-iteration": 0.29783398933832156, "set_robot_commands": 0.002225082882201474, "deviation-center-line": 0.32626753243489137, "driven_lanedir_consec": 0.45976443035215064, "sim_compute_sim_state": 0.011188541328050814, "sim_compute_performance-ego0": 0.0020106181255361653}, "LF-norm-techtrack-000-ego0": {"driven_any": 3.3066922102319163, "get_ui_image": 0.03331195232735486, "step_physics": 0.19791282710553132, "survival_time": 26.00000000000023, "driven_lanedir": 0.8097370505290785, "get_state_dump": 0.00472281349826454, "get_robot_state": 0.003854018484104618, "sim_render-ego0": 0.0038739797478674017, "get_duckie_state": 1.3252595107065738e-06, "in-drivable-lane": 17.600000000000186, "deviation-heading": 4.908403032636656, "agent_compute-ego0": 0.013285402601831475, "complete-iteration": 0.27085868548065595, "set_robot_commands": 0.0022878578208000784, "deviation-center-line": 0.8296264062695263, "driven_lanedir_consec": 0.8097370505290785, "sim_compute_sim_state": 0.009485105177720082, "sim_compute_performance-ego0": 0.002041430024862747}, "LF-norm-small_loop-000-ego0": {"driven_any": 7.919935175412725, "get_ui_image": 0.027477919112434992, "step_physics": 0.14680038066231935, "survival_time": 59.99999999999873, "driven_lanedir": 2.1835728363308498, "get_state_dump": 0.004810240147612077, "get_robot_state": 0.003891033494998573, "sim_render-ego0": 0.003972951220433778, "get_duckie_state": 1.2990934068614696e-06, "in-drivable-lane": 40.39999999999905, "deviation-heading": 8.724287359625125, "agent_compute-ego0": 0.013267622502221356, "complete-iteration": 0.21106568323781744, "set_robot_commands": 0.0023647698633478247, "deviation-center-line": 1.2462828584510095, "driven_lanedir_consec": 2.120572199003452, "sim_compute_sim_state": 0.006269804941029672, "sim_compute_performance-ego0": 0.002124956704297729}}
set_robot_commands_max0.0023647698633478247
set_robot_commands_mean0.0022725928567413796
set_robot_commands_median0.0022564703515007764
set_robot_commands_min0.0022126608606161413
sim_compute_performance-ego0_max0.002124956704297729
sim_compute_performance-ego0_mean0.002041512896820319
sim_compute_performance-ego0_median0.0020260240751994564
sim_compute_performance-ego0_min0.0019890467325846353
sim_compute_sim_state_max0.011188541328050814
sim_compute_sim_state_mean0.008912478607209739
sim_compute_sim_state_median0.009095784079879234
sim_compute_sim_state_min0.006269804941029672
sim_render-ego0_max0.003972951220433778
sim_render-ego0_mean0.0038600113595949497
sim_render-ego0_median0.003835810960033404
sim_render-ego0_min0.0037954722978792137
simulation-passed1
step_physics_max0.22024835012235697
step_physics_mean0.1791078165262502
step_physics_median0.17469126766016227
step_physics_min0.14680038066231935
survival_time_max59.99999999999873
survival_time_mean31.949999999999804
survival_time_min8.999999999999993
No reset possible
54380LFv-simfailedyes0:17:15
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 218, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 304, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 341, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "ego0".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 563, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 135, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 281, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/ego0-out after 2045 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 286, in main
    length_s = await run_episode(
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 567, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible