Gradient-free learning of a closed-loop wall controller for turbulent drag reduction
On the map: Deep reinforcement learning methods → RL with diffusion and flow models
Abstract (arXiv)
Closed-loop wall controllers learnt by multi-agent reinforcement learning are usually trained on periodic boxes far smaller than the flows they are meant to drive, and a large part of their drag reduction is lost when they are carried across. Retraining on the target domain is not an affordable remedy: the centralised critic that assigns credit to each wall patch degrades as patches are added, the zero-net-mass constraint couples the patches it is asked to separate, and the episodes must be collected in sequence at a cost that grows with the domain. We propose instead a short gradient-free refinement stage, applied to the transferred policy on the domain it will drive. An evolution strategy scores whole flow episodes against a regularised objective, so no credit has to be assigned to individual patches, and the candidates in a generation are independent and run in parallel. Applied to a recurrent multi-agent policy trained on a minimal flow unit at Retausimeq180 and evaluated on a channel sixteen times larger in wall-parallel area, a few generations raise the drag reduction from 19.0% to 25.7%, above the 22.5% of opposition control. Because the policies before and after refinement share an architecture and a training history, the difference between the controlled flows follows from the refinement alone. It shows in the friction decomposition, in the Reynolds stresses and in the near-wall spectra, and the actuation moves from a weak coupling to the wall-normal velocity towards a strong coupling to the streamwise fluctuation.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL with diffusion and flow models, one of 15 directions in Deep reinforcement learning methods. Builds on 14 of its 58 references that are in the map. First posted on arXiv 14 Jul 2026.
Builds on
14 of 58 references.
Closest ancestors · 1
Same direction, newest first
Other references · 13
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
- 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
- 2014How transferable are features in deep neural networks?cited by 1,225
- 2017Counterfactual Multi-Agent Policy Gradientscited by 751
- 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
- 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 574
- 2019Machine Learning for Fluid Mechanicscited by 385
- 2020Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solverscited by 205
- 2019A review on Deep Reinforcement Learning for Fluid Mechanicscited by 66
- 2021Scientific multi-agent reinforcement learning for wall-models of turbulent flowscited by 52
- 2023Effective control of two-dimensional Rayleigh--B'enard convection: invariant multi-agent reinforcement learning is all you needcited by 27
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Explainable deep reinforcement learning reveals energy-efficient control strategies for turbulent drag reduction
- 2025Physics-informed Neural-operator Predictive Control for Drag Reduction in Turbulent Flows
- 2026Physics-guided surrogate learning enables zero-shot control of turbulent wings
- 2025Invariant Control Strategies for Active Flow Control using Graph Neural Networks
- 2026A Blueprint for Self-Evolving Coding Agents in Vehicle Aerodynamic Drag Prediction
- 2025SAC Flow: Sample-Efficient Reinforcement Learning of Flow-Based Policies via Velocity-Reparameterized Sequential Modeling
- 2026Heuristic Learning for Active Flow Control Using Coding Agents
- 2025Convergence of Flow-Policy Gradient Learning for Linear Quadratic Regulator Problems