# Optimal navigation of magnetic artificial microswimmers in blood capillaries with deep reinforcement learning

arXiv: https://arxiv.org/abs/2404.02171 · HTML: https://mlsanity.com/paper/2404.02171/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 57 references that are in the map.
- First posted on arXiv 29 Mar 2024.

## Builds on (most cited first, 12 of 12)

- [Sim-to-Real Transfer of Robotic Control with Dynamics Randomization](https://mlsanity.com/paper/1710.06537/) (2017)
- [Efficient collective swimming by harnessing vortices through deep reinforcement learning](https://mlsanity.com/paper/1802.02674/) (2018)
- [Remember and Forget for Experience Replay](https://mlsanity.com/paper/1807.05827/) (2018)
- [Flow Navigation by Smart Microswimmers via Reinforcement Learning](https://mlsanity.com/paper/1701.08848/) (2017)
- [Virtual vs. Real: Trading Off Simulations and Physical Experiments in Reinforcement Learning with Bayesian Optimization](https://mlsanity.com/paper/1703.01250/) (2017)
- [Zermelo's problem: Optimal point-to-point navigation in 2D turbulent flows using Reinforcement Learning](https://mlsanity.com/paper/1907.08591/) (2019)
- [Machine learning strategies for path-planning microswimmers in turbulent flows](https://mlsanity.com/paper/1910.01728/) (2019)
- [Reinforcement learning for pursuit and evasion of microswimmers at low Reynolds number](https://mlsanity.com/paper/2106.08609/) (2021)
- [Korali: Efficient and Scalable Software Framework for Bayesian Uncertainty Quantification and Stochastic Optimization](https://mlsanity.com/paper/2005.13457/) (2020)
- [Learning swimming escape patterns for larval fish under energy constraints](https://mlsanity.com/paper/2105.00771/) (2021)
- [The stress-free state of human erythrocytes: data driven inference of a transferable RBC model](https://mlsanity.com/paper/2303.03404/) (2023)
- [Mirheo: High-Performance Mesoscale Simulations for Microfluidics](https://mlsanity.com/paper/1911.04712/) (2019)

## Built on by (most cited first, 1 of 1)

- [Reinforcement Learning for Active Matter](https://mlsanity.com/paper/2503.23308/) (2025)

## Abstract (arXiv)

Biomedical applications such as targeted drug delivery, microsurgery, and sensing rely on reaching precise areas within the body in a minimally invasive way. Artificial bacterial flagella (ABFs) have emerged as potential tools for this task by navigating through the circulatory system with the help of external magnetic fields. While their swimming characteristics are well understood in simple settings, their controlled navigation through realistic capillary networks remains a significant challenge due to the complexity of blood flow and the high computational cost of detailed simulations. We address this challenge by conducting numerical simulations of ABFs in retinal capillaries, propelled by an external magnetic field. The simulations are based on a validated blood model that predicts the dynamics of individual red blood cells and their hydrodynamic interactions with ABFs. The magnetic field follows a control policy that brings the ABF to a prescribed target. The control policy is learned with an actor-critic, off-policy reinforcement learning algorithm coupled with a reduced-order model of the system. We show that the same policy robustly guides the ABF to a prescribed target in both the reduced-order model and the fine-grained blood simulations. This approach is suitable for designing robust control policies for personalized medicine at moderate computational cost.