Duckietown Challenges Home Challenges Submissions

Evaluator 1991

ID1991
evaluatorreg10-05e8858c90a0-1
ownerI don't have one 😀
machinearchimede
processreg10-05e8858c90a0-1
versiond-c:6.1.5;d-c-r:6.0.31
first heard
last heard
statusinactive
# evaluating
# success2 34856
# timeout
# failed2 34854
# error
# aborted
# host-error
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
348719394Jean-Sébastien Grondin 🇨🇦sim-exercise-1aido5-LF-sim-validationLFv-simsuccessnoreg10-05e8858c90a0-10:27:50
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.997822328107696
survival_time_median14.950000000000076
deviation-center-line_median0.5707520215771001
in-drivable-lane_median4.350000000000062


other stats
agent_compute-ego_max0.06052375555038452
agent_compute-ego_mean0.05494725049140238
agent_compute-ego_median0.05752650896708171
agent_compute-ego_min0.02879940762239344
complete-iteration_max0.39008822679519656
complete-iteration_mean0.36305825465425046
complete-iteration_median0.36315324346224465
complete-iteration_min0.3163910388946533
deviation-center-line_max1.1019894192863415
deviation-center-line_mean0.5783263996836601
deviation-center-line_min0.2635938558377634
deviation-heading_max4.941740008456173
deviation-heading_mean1.4574802046232518
deviation-heading_median1.0893007503604446
deviation-heading_min0.5548539048382177
driven_any_max3.1539100357743415
driven_any_mean2.623637700730886
driven_any_median2.835536205487719
driven_any_min1.5959963716998475
driven_lanedir_consec_max2.5719066196111933
driven_lanedir_consec_mean1.9344810176934124
driven_lanedir_consec_min1.3458074472416626
driven_lanedir_max2.5719066196111933
driven_lanedir_mean1.9344810176934124
driven_lanedir_median1.997822328107696
driven_lanedir_min1.3458074472416626
get_state_dump_max0.016528666814168293
get_state_dump_mean0.016166633920881184
get_state_dump_median0.016191167200312894
get_state_dump_min0.015768725872039795
get_ui_image_max0.07059284369150798
get_ui_image_mean0.06597112261267829
get_ui_image_median0.06543329278628032
get_ui_image_min0.06100537356208353
in-drivable-lane_max6.900000000000098
in-drivable-lane_mean3.775000000000047
in-drivable-lane_min0.4499999999999984
per-episodes
details{"ETHZ_autolab_technical_track-0-0-ego": {"driven_any": 2.065318643580443, "get_ui_image": 0.06100537356208353, "step_physics": 0.13928987839642692, "survival_time": 8.499999999999986, "driven_lanedir": 1.3458074472416626, "get_state_dump": 0.016170684029074275, "sim_render-ego": 0.007649036014781278, "in-drivable-lane": 2.849999999999998, "agent_compute-ego": 0.02879940762239344, "deviation-heading": 0.5548539048382177, "complete-iteration": 0.3163910388946533, "set_robot_commands": 0.0067999573314891145, "deviation-center-line": 0.30429731820930606, "driven_lanedir_consec": 1.3458074472416626, "sim_compute_sim_state": 0.02886813949136173, "sim_compute_performance-ego": 0.006225395202636719, "sim_compute_robot_state-ego": 0.010901180435629453, "sim_compute_robot_state-parked0": 0.010491157980526194}, "ETHZ_autolab_technical_track-0-1-ego": {"driven_any": 2.547702877484277, "get_ui_image": 0.06394489685694377, "step_physics": 0.14982070525487265, "survival_time": 14.950000000000076, "driven_lanedir": 1.596332741133068, "get_state_dump": 0.016482133070627847, "sim_render-ego": 0.0075922425587972, "in-drivable-lane": 4.95000000000007, "agent_compute-ego": 0.05817479530970256, "deviation-heading": 1.1509985448779343, "complete-iteration": 0.3585105268160502, "set_robot_commands": 0.006389908790588379, "deviation-center-line": 0.7547268439400994, "driven_lanedir_consec": 1.596332741133068, "sim_compute_sim_state": 0.03005610466003418, "sim_compute_performance-ego": 0.0058607975641886394, "sim_compute_robot_state-ego": 0.010059581597646078, "sim_compute_robot_state-parked0": 0.009948872725168863}, "ETHZ_autolab_technical_track-1-0-ego": {"driven_any": 3.068187671544322, "get_ui_image": 0.06341464837392172, "step_physics": 0.1494892168045044, "survival_time": 14.950000000000076, "driven_lanedir": 1.9980632935867595, "get_state_dump": 0.016211650371551513, "sim_render-ego": 0.007756438255310059, "in-drivable-lane": 6.450000000000092, "agent_compute-ego": 0.05742003202438355, "deviation-heading": 0.8225169811050672, "complete-iteration": 0.35519253969192505, "set_robot_commands": 0.00633771022160848, "deviation-center-line": 0.4408939908204797, "driven_lanedir_consec": 1.9980632935867595, "sim_compute_sim_state": 0.02803016424179077, "sim_compute_performance-ego": 0.006322376728057862, "sim_compute_robot_state-ego": 0.010221830209096273, "sim_compute_robot_state-parked0": 0.009801491101582844}, "ETHZ_autolab_technical_track-1-1-ego": {"driven_any": 3.0152675912395472, "get_ui_image": 0.07011941989262899, "step_physics": 0.17325471401214598, "survival_time": 14.950000000000076, "driven_lanedir": 2.1059665399252556, "get_state_dump": 0.01586862246195475, "sim_render-ego": 0.007789147694905599, "in-drivable-lane": 5.150000000000073, "agent_compute-ego": 0.05763298590977987, "deviation-heading": 1.0276029558429547, "complete-iteration": 0.38630670070648193, "set_robot_commands": 0.006080220540364584, "deviation-center-line": 0.5561769807864775, "driven_lanedir_consec": 2.1059665399252556, "sim_compute_sim_state": 0.02940132220586141, "sim_compute_performance-ego": 0.005901505947113037, "sim_compute_robot_state-ego": 0.010171554883321126, "sim_compute_robot_state-parked0": 0.009888416131337485}, "ETHZ_autolab_technical_track-2-0-ego": {"driven_any": 2.73076666485662, "get_ui_image": 0.07059284369150798, "step_physics": 0.1754386361440023, "survival_time": 14.950000000000076, "driven_lanedir": 1.6591015046058817, "get_state_dump": 0.016032752990722657, "sim_render-ego": 0.007456762790679932, "in-drivable-lane": 6.900000000000098, "agent_compute-ego": 0.06052375555038452, "deviation-heading": 1.586989530750715, "complete-iteration": 0.3837195348739624, "set_robot_commands": 0.006057525475819905, "deviation-center-line": 0.5853270623677228, "driven_lanedir_consec": 1.6591015046058817, "sim_compute_sim_state": 0.02171053727467855, "sim_compute_performance-ego": 0.005813132921854655, "sim_compute_robot_state-ego": 0.0102364977200826, "sim_compute_robot_state-parked0": 0.009678163528442384}, "ETHZ_autolab_technical_track-2-1-ego": {"driven_any": 2.9403057461188182, "get_ui_image": 0.06496131579081217, "step_physics": 0.16269356886545816, "survival_time": 14.950000000000076, "driven_lanedir": 2.427637137930006, "get_state_dump": 0.015768725872039795, "sim_render-ego": 0.00743316650390625, "in-drivable-lane": 1.600000000000006, "agent_compute-ego": 0.05988408009211222, "deviation-heading": 4.941740008456173, "complete-iteration": 0.36450653155644736, "set_robot_commands": 0.006318938732147217, "deviation-center-line": 1.1019894192863415, "driven_lanedir_consec": 2.427637137930006, "sim_compute_sim_state": 0.02090861479441325, "sim_compute_performance-ego": 0.005920147895812989, "sim_compute_robot_state-ego": 0.01050179878870646, "sim_compute_robot_state-parked0": 0.009935994148254397}, "ETHZ_autolab_technical_track-3-0-ego": {"driven_any": 1.5959963716998475, "get_ui_image": 0.06216355323791504, "step_physics": 0.14872178840637207, "survival_time": 6.249999999999986, "driven_lanedir": 1.461468065061877, "get_state_dump": 0.01647265434265137, "sim_render-ego": 0.006704677581787109, "in-drivable-lane": 0.4499999999999984, "agent_compute-ego": 0.05668980026245117, "deviation-heading": 0.7766023765797093, "complete-iteration": 0.34460968589782714, "set_robot_commands": 0.006413049697875976, "deviation-center-line": 0.2635938558377634, "driven_lanedir_consec": 1.461468065061877, "sim_compute_sim_state": 0.021498458862304688, "sim_compute_performance-ego": 0.005647735595703125, "sim_compute_robot_state-ego": 0.010120521545410157, "sim_compute_robot_state-parked0": 0.009998945236206057}, "ETHZ_autolab_technical_track-3-1-ego": {"driven_any": 2.9742984508551413, "get_ui_image": 0.07032952149709065, "step_physics": 0.17452170848846435, "survival_time": 14.950000000000076, "driven_lanedir": 2.1809454652097893, "get_state_dump": 0.016528666814168293, "sim_render-ego": 0.007296781539916992, "in-drivable-lane": 5.150000000000073, "agent_compute-ego": 0.05805643955866496, "deviation-heading": 1.3467512518307507, "complete-iteration": 0.39008822679519656, "set_robot_commands": 0.006016680399576823, "deviation-center-line": 0.69339894897711, "driven_lanedir_consec": 2.1809454652097893, "sim_compute_sim_state": 0.03148412783940633, "sim_compute_performance-ego": 0.005849494934082032, "sim_compute_robot_state-ego": 0.010076038837432862, "sim_compute_robot_state-parked0": 0.009748744169871012}, "ETHZ_autolab_technical_track-4-0-ego": {"driven_any": 3.1539100357743415, "get_ui_image": 0.06590526978174846, "step_physics": 0.1551316253344218, "survival_time": 14.950000000000076, "driven_lanedir": 2.5719066196111933, "get_state_dump": 0.01633933146794637, "sim_render-ego": 0.0076715858777364095, "in-drivable-lane": 3.7500000000000537, "agent_compute-ego": 0.05597880601882935, "deviation-heading": 1.4078907480420877, "complete-iteration": 0.361799955368042, "set_robot_commands": 0.006393809318542481, "deviation-center-line": 0.6236426029535609, "driven_lanedir_consec": 2.5719066196111933, "sim_compute_sim_state": 0.02754944086074829, "sim_compute_performance-ego": 0.006181629498799642, "sim_compute_robot_state-ego": 0.010414946873982749, "sim_compute_robot_state-parked0": 0.010049832661946615}, "ETHZ_autolab_technical_track-4-1-ego": {"driven_any": 2.144622954155503, "get_ui_image": 0.06727438344213063, "step_physics": 0.1654098604967494, "survival_time": 8.349999999999984, "driven_lanedir": 1.997581362628632, "get_state_dump": 0.015791117788074974, "sim_render-ego": 0.007632589625741194, "in-drivable-lane": 0.5000000000000036, "agent_compute-ego": 0.05631240256532224, "deviation-heading": 0.9588557439089088, "complete-iteration": 0.3694578059419186, "set_robot_commands": 0.006068168286077991, "deviation-center-line": 0.4592169736577392, "driven_lanedir_consec": 1.997581362628632, "sim_compute_sim_state": 0.02463744500440038, "sim_compute_performance-ego": 0.005957827596607322, "sim_compute_robot_state-ego": 0.010536220973123333, "sim_compute_robot_state-parked0": 0.009664566930896509}}
set_robot_commands_max0.0067999573314891145
set_robot_commands_mean0.006287596879409095
set_robot_commands_median0.006328324476877848
set_robot_commands_min0.006016680399576823
sim_compute_performance-ego_max0.006322376728057862
sim_compute_performance-ego_mean0.005968004388485602
sim_compute_performance-ego_median0.005910826921463013
sim_compute_performance-ego_min0.005647735595703125
sim_compute_robot_state-ego_max0.010901180435629453
sim_compute_robot_state-ego_mean0.010324017186443109
sim_compute_robot_state-ego_median0.010229163964589436
sim_compute_robot_state-ego_min0.010059581597646078
sim_compute_robot_state-parked0_max0.010491157980526194
sim_compute_robot_state-parked0_mean0.009920618461423236
sim_compute_robot_state-parked0_median0.00991220513979594
sim_compute_robot_state-parked0_min0.009664566930896509
sim_compute_sim_state_max0.03148412783940633
sim_compute_sim_state_mean0.02641443552349996
sim_compute_sim_state_median0.027789802551269532
sim_compute_sim_state_min0.02090861479441325
sim_render-ego_max0.007789147694905599
sim_render-ego_mean0.007498242844356203
sim_render-ego_median0.007612416092269197
sim_render-ego_min0.006704677581787109
simulation-passed1
step_physics_max0.1754386361440023
step_physics_mean0.1593771702203418
step_physics_median0.15891259709994
step_physics_min0.13928987839642692
survival_time_max14.950000000000076
survival_time_mean12.77500000000005
survival_time_min6.249999999999986
No reset possible
348589385Jean-Sébastien Grondin 🇨🇦sim-exercise-1aido5-LF-sim-validationLFv-simfailednoreg10-05e8858c90a0-10:07:04
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 216, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 299, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 337, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "ego".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "experiment_manager.py", line 443, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 133, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 279, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/ego-out after 556 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 666, in scoring_context
    yield cie
  File "experiment_manager.py", line 607, in <module>
    wrap(cie)
  File "experiment_manager.py", line 595, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 213, in main
    length_s = await run_episode(
  File "experiment_manager.py", line 447, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
348569384Jean-Sébastien Grondin 🇨🇦sim-exercise-1aido5-LF-sim-validationLFv-simsuccessnoreg10-05e8858c90a0-10:25:00
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.0
survival_time_median14.950000000000076
deviation-center-line_median0.4054131312023622
in-drivable-lane_median0.0


other stats
agent_compute-ego_max0.03102914253870646
agent_compute-ego_mean0.02989531207084656
agent_compute-ego_median0.02977236032485962
agent_compute-ego_min0.029158381621042888
complete-iteration_max0.288642942905426
complete-iteration_mean0.2692407729625702
complete-iteration_median0.2661059816678365
complete-iteration_min0.25980214595794676
deviation-center-line_max0.6863069918481528
deviation-center-line_mean0.41778364329376794
deviation-center-line_min0.13942680180416853
deviation-heading_max3.7258518640401217
deviation-heading_mean1.8869656519179927
deviation-heading_median1.6198720948235377
deviation-heading_min0.486072248607516
driven_any_max1.3686924727581474e-13
driven_any_mean8.215966626878722e-14
driven_any_median1.3278267374516872e-13
driven_any_min0.0
driven_lanedir_consec_max1.7821788787841797e-05
driven_lanedir_consec_mean3.564357757568359e-06
driven_lanedir_consec_min0.0
driven_lanedir_max1.7821788787841797e-05
driven_lanedir_mean3.564357757568359e-06
driven_lanedir_median0.0
driven_lanedir_min0.0
get_state_dump_max0.02077556848526001
get_state_dump_mean0.020354549884796146
get_state_dump_median0.020394830306371053
get_state_dump_min0.01992011864980062
get_ui_image_max0.05038559913635254
get_ui_image_mean0.04975923601786296
get_ui_image_median0.049944048325220744
get_ui_image_min0.048369191487630206
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"ETHZ_autolab_technical_track-0-0-ego": {"driven_any": 1.328474932138796e-13, "get_ui_image": 0.05013994137446086, "step_physics": 0.09749117453893026, "survival_time": 14.950000000000076, "driven_lanedir": 0.0, "get_state_dump": 0.02045085350672404, "sim_render-ego": 0.008902812004089355, "in-drivable-lane": 0.0, "agent_compute-ego": 0.030131235122680664, "deviation-heading": 1.6198720948235377, "complete-iteration": 0.27040269851684573, "set_robot_commands": 0.008110715548197428, "deviation-center-line": 0.20880585936578705, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.021823510328928628, "sim_compute_performance-ego": 0.007546406586964925, "sim_compute_robot_state-ego": 0.013042558828989664, "sim_compute_robot_state-parked0": 0.012500418027242026}, "ETHZ_autolab_technical_track-0-1-ego": {"driven_any": 1.328474932138796e-13, "get_ui_image": 0.050333110491434734, "step_physics": 0.09597646554311116, "survival_time": 14.950000000000076, "driven_lanedir": 0.0, "get_state_dump": 0.020338807106018063, "sim_render-ego": 0.00905238151550293, "in-drivable-lane": 0.0, "agent_compute-ego": 0.029887340068817134, "deviation-heading": 1.6198720948235377, "complete-iteration": 0.2688682198524475, "set_robot_commands": 0.00826294740041097, "deviation-center-line": 0.20880585936578705, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.0212453556060791, "sim_compute_performance-ego": 0.007625409762064616, "sim_compute_robot_state-ego": 0.013288005193074545, "sim_compute_robot_state-parked0": 0.01260024070739746}, "ETHZ_autolab_technical_track-1-0-ego": {"driven_any": 1.3686924727581474e-13, "get_ui_image": 0.05000580708185832, "step_physics": 0.09680487235387168, "survival_time": 14.950000000000076, "driven_lanedir": 0.0, "get_state_dump": 0.02058799902598063, "sim_render-ego": 0.00855867067972819, "in-drivable-lane": 0.0, "agent_compute-ego": 0.0296144962310791, "deviation-heading": 3.7258518640401217, "complete-iteration": 0.26578744808832805, "set_robot_commands": 0.007901848951975505, "deviation-center-line": 0.13942680180416853, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.019824840227762857, "sim_compute_performance-ego": 0.007230010032653809, "sim_compute_robot_state-ego": 0.012628571192423502, "sim_compute_robot_state-parked0": 0.012372140884399412}, "ETHZ_autolab_technical_track-1-1-ego": {"driven_any": 1.3686924727581474e-13, "get_ui_image": 0.04988228956858317, "step_physics": 0.09622232675552368, "survival_time": 14.950000000000076, "driven_lanedir": 0.0, "get_state_dump": 0.02077556848526001, "sim_render-ego": 0.008847362995147705, "in-drivable-lane": 0.0, "agent_compute-ego": 0.030210795402526858, "deviation-heading": 3.7258518640401217, "complete-iteration": 0.26642451524734495, "set_robot_commands": 0.007949204444885253, "deviation-center-line": 0.13942680180416853, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.019634420871734615, "sim_compute_performance-ego": 0.00726246992746989, "sim_compute_robot_state-ego": 0.012790129979451498, "sim_compute_robot_state-parked0": 0.012584366003672284}, "ETHZ_autolab_technical_track-2-0-ego": {"driven_any": 0.0, "get_ui_image": 0.048369191487630206, "step_physics": 0.09532676060994466, "survival_time": 14.950000000000076, "driven_lanedir": 1.7821788787841797e-05, "get_state_dump": 0.02055429220199585, "sim_render-ego": 0.008772498766581217, "in-drivable-lane": 0.0, "agent_compute-ego": 0.03102914253870646, "deviation-heading": 0.8029087130506755, "complete-iteration": 0.25980214595794676, "set_robot_commands": 0.007735409736633301, "deviation-center-line": 0.4054131312023622, "driven_lanedir_consec": 1.7821788787841797e-05, "sim_compute_sim_state": 0.016042664845784507, "sim_compute_performance-ego": 0.007095662752787272, "sim_compute_robot_state-ego": 0.012501813570658368, "sim_compute_robot_state-parked0": 0.012107664744059244}, "ETHZ_autolab_technical_track-2-1-ego": {"driven_any": 0.0, "get_ui_image": 0.049771281083424886, "step_physics": 0.097293438911438, "survival_time": 14.950000000000076, "driven_lanedir": 1.7821788787841797e-05, "get_state_dump": 0.020480222702026367, "sim_render-ego": 0.008907442092895507, "in-drivable-lane": 0.0, "agent_compute-ego": 0.03054646571477254, "deviation-heading": 0.8029087130506755, "complete-iteration": 0.26417272408803305, "set_robot_commands": 0.007876778443654378, "deviation-center-line": 0.4054131312023622, "driven_lanedir_consec": 1.7821788787841797e-05, "sim_compute_sim_state": 0.016708277861277262, "sim_compute_performance-ego": 0.007325336933135986, "sim_compute_robot_state-ego": 0.012788735230763753, "sim_compute_robot_state-parked0": 0.01220442056655884}, "ETHZ_autolab_technical_track-3-0-ego": {"driven_any": 8.298917109073045e-15, "get_ui_image": 0.048856877485911054, "step_physics": 0.11616052468617756, "survival_time": 14.950000000000076, "driven_lanedir": 0.0, "get_state_dump": 0.01992011864980062, "sim_render-ego": 0.008475526968638103, "in-drivable-lane": 0.0, "agent_compute-ego": 0.029158381621042888, "deviation-heading": 0.486072248607516, "complete-iteration": 0.28172140280405683, "set_robot_commands": 0.00767722765604655, "deviation-center-line": 0.6863069918481528, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.019423211415608724, "sim_compute_performance-ego": 0.007191917101542155, "sim_compute_robot_state-ego": 0.01244301478068034, "sim_compute_robot_state-parked0": 0.012165613969167072}, "ETHZ_autolab_technical_track-3-1-ego": {"driven_any": 8.298917109073045e-15, "get_ui_image": 0.050081689357757565, "step_physics": 0.11958998521169029, "survival_time": 14.950000000000076, "driven_lanedir": 0.0, "get_state_dump": 0.019986608028411863, "sim_render-ego": 0.008676045735677084, "in-drivable-lane": 0.0, "agent_compute-ego": 0.0296573805809021, "deviation-heading": 0.486072248607516, "complete-iteration": 0.288642942905426, "set_robot_commands": 0.007981605529785156, "deviation-center-line": 0.6863069918481528, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.01984152873357137, "sim_compute_performance-ego": 0.007274377346038819, "sim_compute_robot_state-ego": 0.012930893103281656, "sim_compute_robot_state-parked0": 0.012370884418487549}, "ETHZ_autolab_technical_track-4-0-ego": {"driven_any": 1.3278267374516872e-13, "get_ui_image": 0.05038559913635254, "step_physics": 0.09764926036198934, "survival_time": 14.950000000000076, "driven_lanedir": 0.0, "get_state_dump": 0.02015842835108439, "sim_render-ego": 0.008663360277811687, "in-drivable-lane": 0.0, "agent_compute-ego": 0.029276862939198813, "deviation-heading": 2.800123339068113, "complete-iteration": 0.2647268509864807, "set_robot_commands": 0.007989675203959146, "deviation-center-line": 0.6489654322483692, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.017829991976420084, "sim_compute_performance-ego": 0.007288517951965332, "sim_compute_robot_state-ego": 0.012839601039886475, "sim_compute_robot_state-parked0": 0.012391076882680258}, "ETHZ_autolab_technical_track-4-1-ego": {"driven_any": 1.3278267374516872e-13, "get_ui_image": 0.049766573111216225, "step_physics": 0.09594684600830078, "survival_time": 14.950000000000076, "driven_lanedir": 0.0, "get_state_dump": 0.020292600790659587, "sim_render-ego": 0.008667901357014974, "in-drivable-lane": 0.0, "agent_compute-ego": 0.029441020488739013, "deviation-heading": 2.800123339068113, "complete-iteration": 0.2618587811787923, "set_robot_commands": 0.007850963274637858, "deviation-center-line": 0.6489654322483692, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.01749794403711955, "sim_compute_performance-ego": 0.0071576579411824545, "sim_compute_robot_state-ego": 0.012815848191579184, "sim_compute_robot_state-parked0": 0.012165342966715497}}
set_robot_commands_max0.00826294740041097
set_robot_commands_mean0.007933637619018554
set_robot_commands_median0.007925526698430378
set_robot_commands_min0.00767722765604655
sim_compute_performance-ego_max0.007625409762064616
sim_compute_performance-ego_mean0.007299776633580525
sim_compute_performance-ego_median0.007268423636754354
sim_compute_performance-ego_min0.007095662752787272
sim_compute_robot_state-ego_max0.013288005193074545
sim_compute_robot_state-ego_mean0.012806917111078895
sim_compute_robot_state-ego_median0.01280298908551534
sim_compute_robot_state-ego_min0.01244301478068034
sim_compute_robot_state-parked0_max0.01260024070739746
sim_compute_robot_state-parked0_mean0.012346216917037965
sim_compute_robot_state-parked0_median0.01237151265144348
sim_compute_robot_state-parked0_min0.012107664744059244
sim_compute_sim_state_max0.021823510328928628
sim_compute_sim_state_mean0.018987174590428667
sim_compute_sim_state_median0.01952881614367167
sim_compute_sim_state_min0.016042664845784507
sim_render-ego_max0.00905238151550293
sim_render-ego_mean0.008752400239308676
sim_render-ego_median0.00872427225112915
sim_render-ego_min0.008475526968638103
simulation-passed1
step_physics_max0.11958998521169029
step_physics_mean0.10084616549809776
step_physics_median0.09704915563265484
step_physics_min0.09532676060994466
survival_time_max14.950000000000076
survival_time_mean14.950000000000076
survival_time_min14.950000000000076
No reset possible
348549383Jean-Sébastien Grondin 🇨🇦sim-exercise-1aido5-LF-sim-validationLFv-simfailednoreg10-05e8858c90a0-10:05:12
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 216, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 299, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 337, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "ego".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "experiment_manager.py", line 443, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 133, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 279, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/ego-out after 233 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 666, in scoring_context
    yield cie
  File "experiment_manager.py", line 607, in <module>
    wrap(cie)
  File "experiment_manager.py", line 595, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 213, in main
    length_s = await run_episode(
  File "experiment_manager.py", line 447, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible