# Single-step deep reinforcement learning for open-loop control of laminar and turbulent flows

arXiv: https://arxiv.org/abs/2006.02979 · HTML: https://mlsanity.com/paper/2006.02979/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Jun 2020.

## Built on by (most cited first, 14 of 14)

- [Super-Resolution Analysis via Machine Learning: A Survey for Fluid Flows](https://mlsanity.com/paper/2301.10937/) (2023)
- [Physics-informed Dyna-Style Model-Based Deep Reinforcement Learning for Dynamic Control](https://mlsanity.com/paper/2108.00128/) (2021)
- [Multi-fidelity reinforcement learning framework for shape optimization](https://mlsanity.com/paper/2202.11170/) (2022)
- [How to Control Hydrodynamic Force on Fluidic Pinball via Deep Reinforcement Learning](https://mlsanity.com/paper/2304.11526/) (2023)
- [Theta-Resonance: A Single-Step Reinforcement Learning Method for Design Space Exploration](https://mlsanity.com/paper/2211.02052/) (2022)
- [Inverse Design of Grating Couplers Using the Policy Gradient Method from Reinforcement Learning](https://mlsanity.com/paper/2107.00088/) (2021)
- [Low-Latency Event-Based Velocimetry for Quadrotor Control in a Narrow Pipe](https://mlsanity.com/paper/2507.15444/) (2025)
- [Active Control of Flow over Rotating Cylinder by Multiple Jets using Deep Reinforcement Learning](https://mlsanity.com/paper/2307.12083/) (2023)
- [Asynchronous Parallel Reinforcement Learning for Optimizing Propulsive Performance in Fin Ray Control](https://mlsanity.com/paper/2401.11349/) (2024)
- [Enhancing Quantum Memory Lifetime with Measurement-Free Local Error Correction and Reinforcement Learning](https://mlsanity.com/paper/2408.09524/) (2024)
- [CORE: Constraint-Aware One-Step Reinforcement Learning for Simulation-Guided Neural Network Accelerator Design](https://mlsanity.com/paper/2506.03474/) (2025)
- [Timeliness-Fidelity Tradeoff in 3D Scene Representations](https://mlsanity.com/paper/2407.16575/) (2024)
- [Inverse Design in Distributed Circuits Using Single-Step Reinforcement Learning](https://mlsanity.com/paper/2506.08029/) (2025)
- [Task-Oriented Communications for 3D Scene Representation: Balancing Timeliness and Fidelity](https://mlsanity.com/paper/2509.17282/) (2025)

## Abstract (arXiv)

This research gauges the ability of deep reinforcement learning (DRL) techniques to assist the optimization and control of fluid mechanical systems. It combines a novel, "degenerate" version of the proximal policy optimization (PPO) algorithm, that trains a neural network in optimizing the system only once per learning episode, and an in-house stabilized finite elements environment implementing the variational multiscale (VMS) method, that computes the numerical reward fed to the neural network. Three prototypical examples of separated flows in two dimensions are used as testbed for developing the methodology, each of which adds a layer of complexity due either to the unsteadiness of the flow solutions, or the sharpness of the objective function, or the dimension of the control parameter space. Relevance is carefully assessed by comparing systematically to reference data obtained by canonical direct and adjoint methods. Beyond adding value to the shallow literature on this subject, these findings establish the potential of single-step PPO for reliable black-box optimization of computational fluid dynamics (CFD) systems, which paves the way for future progress in optimal flow control using this new class of methods.