The Pulse
Researchers Train Microrobot Navigation Policies in Under 10 Minutes
A vectorized simulator processed roughly 190,000 transitions per second across thousands of parallel environments, allowing researchers to train navigation policies in under 10 minutes. The team also tested zero-shot transfer on two magneti
AI.info Team · Written with AI · fact-checked ·
A simulator processing roughly 190,000 transitions per second let researchers train microrobot navigation policies in under 10 minutes, according to a paper published September 28 in Nature Machine Intelligence. The system contains more than 10,000 artificial vascular environments and parallelizes calculations across thousands of environments. The work targets a bottleneck in deep reinforcement learning for microrobots: training that can take hours or days.
Thousands of environments running in parallel
The researchers built a fully vectorized simulator around a dataset of 13,278 artificial vascular environments. At each simulation step, it updates microrobot dynamics, extracts visual features using a LiDAR-inspired ray-casting method, checks whether movements are feasible, and calculates observations and rewards. In the paper’s abstract, the authors write:
“Results show that the proposed learning framework reduces training time to under 10 minutes, while supporting zero-shot deployment across distinct microrobot types and navigation scenarios.”
The system’s throughput depends on parallelism: in the paper’s selected training setup, it used 4,096 parallel environments. The authors also tested configurations with up to 8,192 parallel environments; one configuration reached 228,117 transitions per second. These are parallel simulation instances, not claims that all 13,278 environments run at once.
The speed is paired with a reward design called task-shaping-regularization, or TSR. It combines task signals for reaching a target or failing with feedback about progress and movement quality. The paper distinguishes the roles of those signals: progress shaping helps the policy learn faster, while regularization favors smoother actions and greater obstacle clearance. Across four evaluated scenes, the authors report action-smoothness improvements of 33.7% to 34.9% and obstacle-safety improvements of 2.1% to 2.5% when regularization was added.
Testing transfer on physical microrobots
The study reports sim-to-real tests on two magnetically actuated devices: a roughly 20-micrometre pollen-based microparticle and a helical microrobot. Without task-specific retraining, the learned policy guided both through artificial vascular channels in previously unseen scenarios. The paper also reports navigation among 11 moving obstacles, which were not part of the training scenarios.
In a separate flow test, the helical microrobot moved through a 5-millimetre-diameter channel against fluid flowing at 2 millimetres per second. The flow slowed and destabilized its movement compared with still fluid, but the policy remained effective, the authors report. The paper says closed-loop control ran at an average rate of 45 hertz.
The authors also tested navigation in three dimensions. One demonstration used an open workspace with two 3D-printed obstacles. Another used a real-size, tortuous phantom of the human cerebral artery; supplementary material also shows navigation in a human brain vasculature model. These are model and phantom tests, not evidence of navigation in a living person’s blood vessels or clinical use.
Artificial training worlds, practical tests
The simulator’s large training set consists of artificial channels designed to capture features such as changes in vessel diameter, branching and curved boundaries. The paper separately describes tests in vascular channels extracted from retinal fundus images, as well as the physical-robot demonstrations. Keeping those settings distinct matters: the training result demonstrates faster policy learning in simulation, while the robot and phantom experiments test how a learned policy transfers to selected physical environments.
The potential benefit is a shorter engineering cycle: researchers can adjust a control policy or reward design and retrain in minutes rather than hours or days. The study demonstrates transfer across two microrobot types and several test conditions, but does not establish that such a system can safely navigate living human vessels. The paper makes its code available through GitHub and reports that the vascular dataset is available through Zenodo.
Yinghan Sun and co-authors, Nature Machine Intelligence