# Flow Navigation by Smart Microswimmers via Reinforcement Learning

arXiv: https://arxiv.org/abs/1701.08848 · HTML: https://mlsanity.com/paper/1701.08848/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jan 2017.

## Built on by (most cited first, 30 of 34)

- [Learning Efficient Navigation in Vortical Flow Fields](https://mlsanity.com/paper/2102.10536/) (2021)
- [SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learning](https://mlsanity.com/paper/2403.09110/) (2024)
- [Symmetry reduction for deep reinforcement learning active control of chaotic spatiotemporal dynamics](https://mlsanity.com/paper/2104.05437/) (2021)
- [Koopman-Assisted Reinforcement Learning](https://mlsanity.com/paper/2403.02290/) (2024)
- [Reinforcement learning for pursuit and evasion of microswimmers at low Reynolds number](https://mlsanity.com/paper/2106.08609/) (2021)
- [Optimal control of point-to-point navigation in turbulent time-dependent flows using Reinforcement Learning](https://mlsanity.com/paper/2103.00329/) (2021)
- [Fast Aquatic Swimmer Optimization with Differentiable Projective Dynamics and Neural Network Hydrodynamic Models](https://mlsanity.com/paper/2204.12584/) (2022)
- [Reinforcement Learning for Active Matter](https://mlsanity.com/paper/2503.23308/) (2025)
- [Enhancing Efficiency and Propulsion in Bio-mimetic Robotic Fish through End-to-End Deep Reinforcement Learning](https://mlsanity.com/paper/2506.04627/) (2025)
- [Learning swimming escape patterns for larval fish under energy constraints](https://mlsanity.com/paper/2105.00771/) (2021)
- [Smart Magnetic Microrobots Learn to Swim with Deep Reinforcement Learning](https://mlsanity.com/paper/2201.05599/) (2022)
- [Harnessing Synthetic Active Particles for Physical Reservoir Computing](https://mlsanity.com/paper/2307.15010/) (2023)
- [Learning Adaptive Hydrodynamic Models Using Neural ODEs in Complex Conditions](https://mlsanity.com/paper/2410.00490/) (2024)
- [Emergent behavior and neural dynamics in artificial agents tracking turbulent plumes](https://mlsanity.com/paper/2109.12434/) (2021)
- [Training microrobots to swim by a large language model](https://mlsanity.com/paper/2402.00044/) (2024)
- [Chemotaxis of sea urchin sperm cells through deep reinforcement learning](https://mlsanity.com/paper/2209.07407/) (2022)
- [Learning to swim efficiently in a nonuniform flow field](https://mlsanity.com/paper/2212.11482/) (2022)
- [Deep-learning-powered data analysis in plankton ecology](https://mlsanity.com/paper/2309.08500/) (2023)
- [Learning optimal integration of spatial and temporal information in noisy chemotaxis](https://mlsanity.com/paper/2310.10531/) (2023)
- [Asynchronous Parallel Reinforcement Learning for Optimizing Propulsive Performance in Fin Ray Control](https://mlsanity.com/paper/2401.11349/) (2024)
- [Optimal navigation of magnetic artificial microswimmers in blood capillaries with deep reinforcement learning](https://mlsanity.com/paper/2404.02171/) (2024)
- [Navigation of a Three-Link Microswimmer via Deep Reinforcement Learning](https://mlsanity.com/paper/2506.00084/) (2025)
- [A study of first-passage time minimization via Q-learning in heated gridworlds](https://mlsanity.com/paper/2110.02129/) (2021)
- [Stabilising viscous extensional flows using Reinforcement Learning](https://mlsanity.com/paper/2110.14677/) (2021)
- [Classifying Turbulent Environments via Machine Learning](https://mlsanity.com/paper/2201.00732/) (2022)
- [Policy composition in reinforcement learning via multi-objective policy optimization](https://mlsanity.com/paper/2308.15470/) (2023)
- [A critical assessment of reinforcement learning methods for microswimmer navigation in complex flows](https://mlsanity.com/paper/2505.05525/) (2025)
- [Controlling Topological Defects in Polar Fluids via Reinforcement Learning](https://mlsanity.com/paper/2507.19298/) (2025)
- [Smart navigation of a gravity-driven glider with adjustable centre-of-mass](https://mlsanity.com/paper/2510.09250/) (2025)
- [A perspective on fluid mechanical environments for challenges in reinforcement learning](https://mlsanity.com/paper/2605.25011/) (2026)

## Abstract (arXiv)

Smart active particles can acquire some limited knowledge of the fluid environment from simple mechanical cues and exert a control on their preferred steering direction. Their goal is to learn the best way to navigate by exploiting the underlying flow whenever possible. As an example, we focus our attention on smart gravitactic swimmers. These are active particles whose task is to reach the highest altitude within some time horizon, given the constraints enforced by fluid mechanics. By means of numerical experiments, we show that swimmers indeed learn nearly optimal strategies just by experience. A reinforcement learning algorithm allows particles to learn effective strategies even in difficult situations when, in the absence of control, they would end up being trapped by flow structures. These strategies are highly nontrivial and cannot be easily guessed in advance. This Letter illustrates the potential of reinforcement learning algorithms to model adaptive behavior in complex flows and paves the way towards the engineering of smart microswimmers that solve difficult navigation problems.