# Finding Efficient Swimming Strategies in a Three Dimensional Chaotic Flow by Reinforcement Learning

arXiv: https://arxiv.org/abs/1711.05826 · HTML: https://mlsanity.com/paper/1711.05826/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Nov 2017.

## Built on by (most cited first, 11 of 11)

- [Physics-informed Dyna-Style Model-Based Deep Reinforcement Learning for Dynamic Control](https://mlsanity.com/paper/2108.00128/) (2021)
- [Optimal control of point-to-point navigation in turbulent time-dependent flows using Reinforcement Learning](https://mlsanity.com/paper/2103.00329/) (2021)
- [Harnessing Synthetic Active Particles for Physical Reservoir Computing](https://mlsanity.com/paper/2307.15010/) (2023)
- [Chemotaxis of sea urchin sperm cells through deep reinforcement learning](https://mlsanity.com/paper/2209.07407/) (2022)
- [Learning to swim efficiently in a nonuniform flow field](https://mlsanity.com/paper/2212.11482/) (2022)
- [Deep-learning-powered data analysis in plankton ecology](https://mlsanity.com/paper/2309.08500/) (2023)
- [Asynchronous Parallel Reinforcement Learning for Optimizing Propulsive Performance in Fin Ray Control](https://mlsanity.com/paper/2401.11349/) (2024)
- [Stabilising viscous extensional flows using Reinforcement Learning](https://mlsanity.com/paper/2110.14677/) (2021)
- [A critical assessment of reinforcement learning methods for microswimmer navigation in complex flows](https://mlsanity.com/paper/2505.05525/) (2025)
- [Offline Reinforcement Learning for Fluid Controls: Data-based Multi-observational Policy Extraction](https://mlsanity.com/paper/2606.31025/) (2026)
- [Flow-aware Optimal Navigation in Unsteady Flows through Reinforcement Learning](https://mlsanity.com/paper/2607.13553/) (2026)

## Abstract (arXiv)

We apply a reinforcement learning algorithm to show how smart particles can learn approximately optimal strategies to navigate in complex flows. In this paper we consider microswimmers in a paradigmatic three-dimensional case given by a stationary superposition of two Arnold-Beltrami-Childress flows with chaotic advection along streamlines. In such a flow, we study the evolution of point-like particles which can decide in which direction to swim, while keeping the velocity amplitude constant. We show that it is sufficient to endow the swimmers with a very restricted set of actions (six fixed swimming directions in our case) to have enough freedom to find efficient strategies to move upward and escape local fluid traps. The key ingredient is the learning-from-experience structure of the algorithm, which assigns positive or negative rewards depending on whether the taken action is, or is not, profitable for the predetermined goal in the long term horizon. This is another example supporting the efficiency of the reinforcement learning approach to learn how to accomplish difficult tasks in complex fluid environments.