Duckietown Challenges Home Challenges Submissions

Submission 11568

Submission11568
Competingyes
Challengeaido5-LF-sim-validation
UserDishank Bansal 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 54355
Next
User labelexercise_state_estimation
Admin priority50
Blessingn/a
User priority50

54355

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

See previous jobs for previous versions of challenges
Job IDstepstatusup to datedate starteddate completeddurationmessage
54355LFv-simsuccessyes0:29:13
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median3.5857845252028357
survival_time_median59.99999999999873
deviation-center-line_median2.9410512004800387
in-drivable-lane_median14.899999999999723


other stats
agent_compute-ego0_max0.01291652295574452
agent_compute-ego0_mean0.012288634599552695
agent_compute-ego0_median0.012119794368346863
agent_compute-ego0_min0.011998426705772534
complete-iteration_max0.27150929798869966
complete-iteration_mean0.2295352871195673
complete-iteration_median0.21992763482363
complete-iteration_min0.20677658084230952
deviation-center-line_max3.951015535066344
deviation-center-line_mean2.495808091641886
deviation-center-line_min0.15011443054112292
deviation-heading_max18.526043218679295
deviation-heading_mean12.262728393979543
deviation-heading_median14.418929156680418
deviation-heading_min1.6870120438780396
driven_any_max7.920774788478755
driven_any_mean6.166322729469283
driven_any_median7.916877550312439
driven_any_min0.9107610287735024
driven_lanedir_consec_max5.616576989966457
driven_lanedir_consec_mean3.3206171018899364
driven_lanedir_consec_min0.494322367187618
driven_lanedir_max5.616576989966457
driven_lanedir_mean3.3206171018899364
driven_lanedir_median3.5857845252028357
driven_lanedir_min0.494322367187618
get_duckie_state_max1.5744623148216391e-06
get_duckie_state_mean1.4223292147622763e-06
get_duckie_state_median1.421578222270016e-06
get_duckie_state_min1.271698099687435e-06
get_robot_state_max0.003902555206832441
get_robot_state_mean0.0038545600126621655
get_robot_state_median0.003882054572577128
get_robot_state_min0.003751575698661963
get_state_dump_max0.004868245541702004
get_state_dump_mean0.004800456351640185
get_state_dump_median0.004837461258862559
get_state_dump_min0.0046586573471336145
get_ui_image_max0.037564609035755855
get_ui_image_mean0.03140263842873656
get_ui_image_median0.030454922179000563
get_ui_image_min0.027136100321189252
in-drivable-lane_max36.849999999999184
in-drivable-lane_mean17.487499999999656
in-drivable-lane_min3.2999999999999883
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 7.920774788478755, "get_ui_image": 0.02927036269519053, "step_physics": 0.13830999093289975, "survival_time": 59.99999999999873, "driven_lanedir": 5.616576989966457, "get_state_dump": 0.004828923945621487, "get_robot_state": 0.0038775669148720673, "sim_render-ego0": 0.003933587737325626, "get_duckie_state": 1.4497675963186602e-06, "in-drivable-lane": 13.799999999999748, "deviation-heading": 14.387611109748711, "agent_compute-ego0": 0.011998426705772534, "complete-iteration": 0.20677658084230952, "set_robot_commands": 0.002351620115904288, "deviation-center-line": 3.6290273742981904, "driven_lanedir_consec": 5.616576989966457, "sim_compute_sim_state": 0.01000680891699239, "sim_compute_performance-ego0": 0.0021008440695833306}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.9107610287735024, "get_ui_image": 0.037564609035755855, "step_physics": 0.19270324107236084, "survival_time": 7.89999999999998, "driven_lanedir": 0.494322367187618, "get_state_dump": 0.0048459985721036325, "get_robot_state": 0.00388654223028219, "sim_render-ego0": 0.003992076190012806, "get_duckie_state": 1.5744623148216391e-06, "in-drivable-lane": 3.2999999999999883, "deviation-heading": 1.6870120438780396, "agent_compute-ego0": 0.01291652295574452, "complete-iteration": 0.27150929798869966, "set_robot_commands": 0.0023506542421736806, "deviation-center-line": 0.15011443054112292, "driven_lanedir_consec": 0.494322367187618, "sim_compute_sim_state": 0.011072707626054872, "sim_compute_performance-ego0": 0.002077341079711914}, "LF-norm-techtrack-000-ego0": {"driven_any": 7.913832459648168, "get_ui_image": 0.031639481662810595, "step_physics": 0.15699167731997374, "survival_time": 59.99999999999873, "driven_lanedir": 4.928809521257123, "get_state_dump": 0.0046586573471336145, "get_robot_state": 0.003751575698661963, "sim_render-ego0": 0.0037974793150661193, "get_duckie_state": 1.271698099687435e-06, "in-drivable-lane": 15.999999999999698, "deviation-heading": 18.526043218679295, "agent_compute-ego0": 0.012042475183440884, "complete-iteration": 0.23051740783735877, "set_robot_commands": 0.00218424749414093, "deviation-center-line": 3.951015535066344, "driven_lanedir_consec": 4.928809521257123, "sim_compute_sim_state": 0.013316036958083026, "sim_compute_performance-ego0": 0.002043228760845556}, "LF-norm-small_loop-000-ego0": {"driven_any": 7.919922640976709, "get_ui_image": 0.027136100321189252, "step_physics": 0.14643169997832262, "survival_time": 59.99999999999873, "driven_lanedir": 2.2427595291485485, "get_state_dump": 0.004868245541702004, "get_robot_state": 0.003902555206832441, "sim_render-ego0": 0.00387162233173202, "get_duckie_state": 1.3933888482213717e-06, "in-drivable-lane": 36.849999999999184, "deviation-heading": 14.450247203612124, "agent_compute-ego0": 0.012197113553252844, "complete-iteration": 0.20933786180990124, "set_robot_commands": 0.0023034657169440503, "deviation-center-line": 2.2530750266618864, "driven_lanedir_consec": 2.2427595291485485, "sim_compute_sim_state": 0.006462501348007926, "sim_compute_performance-ego0": 0.0020667567638235225}}
set_robot_commands_max0.002351620115904288
set_robot_commands_mean0.0022974968922907374
set_robot_commands_median0.0023270599795588657
set_robot_commands_min0.00218424749414093
sim_compute_performance-ego0_max0.0021008440695833306
sim_compute_performance-ego0_mean0.0020720426684910807
sim_compute_performance-ego0_median0.0020720489217677185
sim_compute_performance-ego0_min0.002043228760845556
sim_compute_sim_state_max0.013316036958083026
sim_compute_sim_state_mean0.010214513712284552
sim_compute_sim_state_median0.010539758271523631
sim_compute_sim_state_min0.006462501348007926
sim_render-ego0_max0.003992076190012806
sim_render-ego0_mean0.0038986913935341425
sim_render-ego0_median0.003902605034528823
sim_render-ego0_min0.0037974793150661193
simulation-passed1
step_physics_max0.19270324107236084
step_physics_mean0.15860915232588924
step_physics_median0.1517116886491482
step_physics_min0.13830999093289975
survival_time_max59.99999999999873
survival_time_mean46.97499999999904
survival_time_min7.89999999999998
No reset possible
54347LFv-simfailedyes0:25:56
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 218, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 304, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 341, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "ego0".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 563, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 135, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 281, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/ego0-out after 3280 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 286, in main
    length_s = await run_episode(
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 567, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
54343LFv-simsuccessyes0:27:54
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
54340LFv-simsuccessyes0:29:59
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible