# Physics-Guided Actor-Critic Reinforcement Learning for Swimming in Turbulence

arXiv: https://arxiv.org/abs/2406.10242 · HTML: https://mlsanity.com/paper/2406.10242/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RL with diffusion and flow models, one of 15 directions in Deep reinforcement learning methods.
- Builds on 14 of its 35 references that are in the map.
- First posted on arXiv 5 Jun 2024.

## Placed in

- Deep reinforcement learning methods / [RL with diffusion and flow models](https://mlsanity.com/deeprl/rl-with-diffusion-and-flow-models/)

## Builds on (most cited first, 14 of 14)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Jump-Start Reinforcement Learning](https://mlsanity.com/paper/2204.02372/) (2022)
- [Learning values across many orders of magnitude](https://mlsanity.com/paper/1602.07714/) (2016)
- [A review on Deep Reinforcement Learning for Fluid Mechanics](https://mlsanity.com/paper/1908.04127/) (2019)
- [Scientific multi-agent reinforcement learning for wall-models of turbulent flows](https://mlsanity.com/paper/2106.11144/) (2021)
- [Learning Efficient Navigation in Vortical Flow Fields](https://mlsanity.com/paper/2102.10536/) (2021)
- [Zermelo's problem: Optimal point-to-point navigation in 2D turbulent flows using Reinforcement Learning](https://mlsanity.com/paper/1907.08591/) (2019)
- [Machine learning strategies for path-planning microswimmers in turbulent flows](https://mlsanity.com/paper/1910.01728/) (2019)
- [Reinforcement learning for pursuit and evasion of microswimmers at low Reynolds number](https://mlsanity.com/paper/2106.08609/) (2021)
- [Optimal tracking strategies in a turbulent flow](https://mlsanity.com/paper/2305.04677/) (2023)
- [Taming Lagrangian Chaos with Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2212.09612/) (2022)
- [Universality and Control of Fat Tails](https://mlsanity.com/paper/2303.09635/) (2023)

## Built on by (most cited first, 2 of 2)

- [PICT -- A Differentiable, GPU-Accelerated Multi-Block PISO Solver for Simulation-Coupled Learning Tasks in Fluid Dynamics](https://mlsanity.com/paper/2505.16992/) (2025)
- [Generative Stochastic Optimal Transport: Guided Harmonic Path-Integral Diffusion](https://mlsanity.com/paper/2512.11859/) (2025)

## Abstract (arXiv)

Turbulent diffusion causes particles placed in proximity to separate. We investigate the required swimming efforts to maintain an active particle close to its passively advected counterpart. We explore optimally balancing these efforts by developing a novel physics-informed reinforcement learning strategy and comparing it with prescribed control and physics-agnostic reinforcement learning strategies. Our scheme, coined the actor-physicist, is an adaptation of the actor-critic algorithm in which the neural network parameterized critic is replaced with an analytically derived physical heuristic function, the physicist. We validate the proposed physics-informed reinforcement learning approach through extensive numerical experiments in both synthetic BK and more realistic Arnold-Beltrami-Childress flow environments, demonstrating its superiority in controlling particle dynamics when compared to standard reinforcement learning methods.