# Zermelo's problem: Optimal point-to-point navigation in 2D turbulent flows using Reinforcement Learning

arXiv: https://arxiv.org/abs/1907.08591 · HTML: https://mlsanity.com/paper/1907.08591/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jul 2019.

## Built on by (most cited first, 21 of 21)

- [Scientific multi-agent reinforcement learning for wall-models of turbulent flows](https://mlsanity.com/paper/2106.11144/) (2021)
- [Learning Efficient Navigation in Vortical Flow Fields](https://mlsanity.com/paper/2102.10536/) (2021)
- [SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learning](https://mlsanity.com/paper/2403.09110/) (2024)
- [Leveraging arbitrary mobile sensor trajectories with shallow recurrent decoder networks for full-state reconstruction](https://mlsanity.com/paper/2307.11793/) (2023)
- [Koopman-Assisted Reinforcement Learning](https://mlsanity.com/paper/2403.02290/) (2024)
- [Reinforcement learning for pursuit and evasion of microswimmers at low Reynolds number](https://mlsanity.com/paper/2106.08609/) (2021)
- [Optimal control of point-to-point navigation in turbulent time-dependent flows using Reinforcement Learning](https://mlsanity.com/paper/2103.00329/) (2021)
- [Navigation in a simplified Urban Flow through Deep Reinforcement Learning](https://mlsanity.com/paper/2409.17922/) (2024)
- [Real-time optimal control of high-dimensional parametrized systems by deep learning-based reduced order models](https://mlsanity.com/paper/2409.05709/) (2024)
- [Optimizing Airborne Wind Energy with Reinforcement Learning](https://mlsanity.com/paper/2203.14271/) (2022)
- [Taming Lagrangian Chaos with Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2212.09612/) (2022)
- [Physics-Guided Actor-Critic Reinforcement Learning for Swimming in Turbulence](https://mlsanity.com/paper/2406.10242/) (2024)
- [Learning to swim efficiently in a nonuniform flow field](https://mlsanity.com/paper/2212.11482/) (2022)
- [Optimal navigation of magnetic artificial microswimmers in blood capillaries with deep reinforcement learning](https://mlsanity.com/paper/2404.02171/) (2024)
- [A study of first-passage time minimization via Q-learning in heated gridworlds](https://mlsanity.com/paper/2110.02129/) (2021)
- [Classifying Turbulent Environments via Machine Learning](https://mlsanity.com/paper/2201.00732/) (2022)
- [Latent feedback control of distributed systems in multiple scenarios through deep learning-based reduced order models](https://mlsanity.com/paper/2412.09942/) (2024)
- [A critical assessment of reinforcement learning methods for microswimmer navigation in complex flows](https://mlsanity.com/paper/2505.05525/) (2025)
- [Khalasi: Energy-Efficient Navigation for Surface Vehicles in Vortical Flow Fields](https://mlsanity.com/paper/2512.06912/) (2025)
- [Emergence of a Flow-Assisted Casting Strategy for Olfactory Navigation via Memory-Augmented Reinforcement Learning](https://mlsanity.com/paper/2605.18881/) (2026)
- [Multi-agent rendezvous in fluid flows via reinforcement learning](https://mlsanity.com/paper/2606.11274/) (2026)

## Abstract (arXiv)

To find the path that minimizes the time to navigate between two given points in a fluid flow is known as Zermelo's problem. Here, we investigate it by using a Reinforcement Learning (RL) approach for the case of a vessel which has a slip velocity with fixed intensity, Vs , but variable direction and navigating in a 2D turbulent sea. We show that an Actor-Critic RL algorithm is able to find quasi-optimal solutions for both time-independent and chaotically evolving flow configurations. For the frozen case, we also compared the results with strategies obtained analytically from continuous Optimal Navigation (ON) protocols. We show that for our application, ON solutions are unstable for the typical duration of the navigation process, and are therefore not useful in practice. On the other hand, RL solutions are much more robust with respect to small changes in the initial conditions and to external noise, even when V s is much smaller than the maximum flow velocity. Furthermore, we show how the RL approach is able to take advantage of the flow properties in order to reach the target, especially when the steering speed is small.