# Plug-and-Play Benchmarking of Reinforcement Learning Algorithms for Large-Scale Flow Control

arXiv: https://arxiv.org/abs/2601.15015 · HTML: https://mlsanity.com/paper/2601.15015/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RL with diffusion and flow models, one of 15 directions in Deep reinforcement learning methods.
- Builds on 29 of its 89 references that are in the map.
- First posted on arXiv 21 Jan 2026.

## Placed in

- Deep reinforcement learning methods / [RL with diffusion and flow models](https://mlsanity.com/deeprl/rl-with-diffusion-and-flow-models/)

## Builds on (most cited first, 29 of 29)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Deep Reinforcement Learning at the Edge of the Statistical Precipice](https://mlsanity.com/paper/2108.13264/) (2021)
- [Deep Reinforcement Learning that Matters](https://mlsanity.com/paper/1709.06560/) (2017)
- [Temporal Difference Learning for Model Predictive Control](https://mlsanity.com/paper/2203.04955/) (2022)
- [PettingZoo: Gym for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2009.14471/) (2020)
- [Accelerated Policy Learning with Parallel Differentiable Simulation](https://mlsanity.com/paper/2204.07137/) (2022)
- [Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations](https://mlsanity.com/paper/2407.07218/) (2024)
- [A review on Deep Reinforcement Learning for Fluid Mechanics](https://mlsanity.com/paper/1908.04127/) (2019)
- [TorchRL: A data-driven decision-making library for PyTorch](https://mlsanity.com/paper/2306.00577/) (2023)
- [Scientific multi-agent reinforcement learning for wall-models of turbulent flows](https://mlsanity.com/paper/2106.11144/) (2021)
- [Effective control of two-dimensional Rayleigh--B'enard convection: invariant multi-agent reinforcement learning is all you need](https://mlsanity.com/paper/2304.02370/) (2023)
- [SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learning](https://mlsanity.com/paper/2403.09110/) (2024)
- [Solving Continuous Control via Q-learning](https://mlsanity.com/paper/2210.12566/) (2022)
- [Distributed Control of Partial Differential Equations Using Convolutional Reinforcement Learning](https://mlsanity.com/paper/2301.10737/) (2023)
- [Learning to Represent Action Values as a Hypergraph on the Action Vertices](https://mlsanity.com/paper/2010.14680/) (2020)
- [Deep Learning Alternative to Explicit Model Predictive Control for Unknown Nonlinear Systems](https://mlsanity.com/paper/2011.03699/) (2020)
- [Stabilizing Reinforcement Learning in Differentiable Multiphysics Simulation](https://mlsanity.com/paper/2412.12089/) (2024)
- [PDE Control Gym: A Benchmark for Data-Driven Boundary Control of Partial Differential Equations](https://mlsanity.com/paper/2405.11401/) (2024)
- [FLEXI: A high order discontinuous Galerkin framework for hyperbolic-parabolic conservation laws](https://mlsanity.com/paper/1910.02858/) (2019)
- [Controlgym: Large-Scale Control Environments for Benchmarking Reinforcement Learning Algorithms](https://mlsanity.com/paper/2311.18736/) (2023)
- [PICT -- A Differentiable, GPU-Accelerated Multi-Block PISO Solver for Simulation-Coupled Learning Tasks in Fluid Dynamics](https://mlsanity.com/paper/2505.16992/) (2025)
- [Deep Reinforcement Learning for Active Flow Control around a Three-Dimensional Flow-Separated Wing at Re = 1,000](https://mlsanity.com/paper/2509.10195/) (2025)
- [Beacon, a lightweight deep reinforcement learning benchmark library for flow control](https://mlsanity.com/paper/2402.17402/) (2024)
- [Gym-preCICE: Reinforcement Learning Environments for Active Flow Control](https://mlsanity.com/paper/2305.02033/) (2023)
- [Control of Rayleigh-B'enard Convection: Effectiveness of Reinforcement Learning in the Turbulent Regime](https://mlsanity.com/paper/2504.12000/) (2025)
- [Physics-informed Neural-operator Predictive Control for Drag Reduction in Turbulent Flows](https://mlsanity.com/paper/2510.03360/) (2025)
- [An Optical Control Environment for Benchmarking Reinforcement Learning Algorithms](https://mlsanity.com/paper/2203.12114/) (2022)
- [Gym-TORAX: Open-source software for integrating reinforcement learning with plasma control simulators in tokamak research](https://mlsanity.com/paper/2510.11283/) (2025)

## Built on by (most cited first, 2 of 2)

- [Heuristic Learning for Active Flow Control Using Coding Agents](https://mlsanity.com/paper/2607.11565/) (2026)
- [Koopman-based surrogate modeling for reinforcement-learning-control of Rayleigh-Benard convection](https://mlsanity.com/paper/2603.28074/) (2026)

## Abstract (arXiv)

Reinforcement learning (RL) has shown promising results in active flow control (AFC), yet progress in the field remains difficult to assess as existing studies rely on heterogeneous observation and actuation schemes, numerical setups, and evaluation protocols. Current AFC benchmarks attempt to address these issues but heavily rely on external computational fluid dynamics (CFD) solvers, are not fully differentiable, and provide limited 3D and multi-agent support. To overcome these limitations, we introduce FluidGym, the first standalone, fully differentiable benchmark suite for RL in AFC. Built entirely in PyTorch on top of the GPU-accelerated PICT solver, FluidGym runs in a single Python stack, requires no external CFD software, and provides standardized evaluation protocols. We present baseline results with PPO, SAC, DPC, and TD-MPC, and release all environments, datasets, and trained models as public resources. FluidGym enables systematic comparison of control methods, establishes a scalable foundation for future research in learning-based flow control, and is available at github.com/safe-autonomous-systems/fluidgym.