# Scientific multi-agent reinforcement learning for wall-models of turbulent flows

arXiv: https://arxiv.org/abs/2106.11144 · HTML: https://mlsanity.com/paper/2106.11144/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 76 references that are in the map.
- First posted on arXiv 21 Jun 2021.

## Builds on (most cited first, 8 of 8)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [End-to-End Training of Deep Visuomotor Policies](https://mlsanity.com/paper/1504.00702/) (2015)
- [Safe and Efficient Off-Policy Reinforcement Learning](https://mlsanity.com/paper/1606.02647/) (2016)
- [Efficient collective swimming by harnessing vortices through deep reinforcement learning](https://mlsanity.com/paper/1802.02674/) (2018)
- [Remember and Forget for Experience Replay](https://mlsanity.com/paper/1807.05827/) (2018)
- [Deep Neural Networks for Data-Driven Turbulence Models](https://mlsanity.com/paper/1806.04482/) (2018)
- [Zermelo's problem: Optimal point-to-point navigation in 2D turbulent flows using Reinforcement Learning](https://mlsanity.com/paper/1907.08591/) (2019)
- [Automating Turbulence Modeling by Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2005.09023/) (2020)

## Built on by (most cited first, 30 of 52)

- [Enhancing Computational Fluid Dynamics with Machine Learning](https://mlsanity.com/paper/2110.02085/) (2021)
- [Towards Scientific Intelligence: A Survey of LLM-based Scientific Agents](https://mlsanity.com/paper/2503.24047/) (2025)
- [Super-Resolution Analysis via Machine Learning: A Survey for Fluid Flows](https://mlsanity.com/paper/2301.10937/) (2023)
- [The transformative potential of machine learning for experiments in fluid mechanics](https://mlsanity.com/paper/2303.15832/) (2023)
- [SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learning](https://mlsanity.com/paper/2403.09110/) (2024)
- [Distributed Control of Partial Differential Equations Using Convolutional Reinforcement Learning](https://mlsanity.com/paper/2301.10737/) (2023)
- [Improving aircraft performance using machine learning: a review](https://mlsanity.com/paper/2210.11481/) (2022)
- [Deep Reinforcement Learning for Turbulence Modeling in Large Eddy Simulations](https://mlsanity.com/paper/2206.11038/) (2022)
- [Deep Learning Closure Models for Large-Eddy Simulation of Flows around Bluff Bodies](https://mlsanity.com/paper/2208.03498/) (2022)
- [Koopman-Assisted Reinforcement Learning](https://mlsanity.com/paper/2403.02290/) (2024)
- [Emerging trends in machine learning for computational fluid dynamics](https://mlsanity.com/paper/2211.15145/) (2022)
- [Multi-agent reinforcement learning for the control of three-dimensional Rayleigh-B'enard convection](https://mlsanity.com/paper/2407.21565/) (2024)
- [Deep Reinforcement Learning for Computational Fluid Dynamics on HPC Systems](https://mlsanity.com/paper/2205.06502/) (2022)
- [Parametric PDE Control with Deep Reinforcement Learning and Differentiable L0-Sparse Polynomial Policies](https://mlsanity.com/paper/2403.15267/) (2024)
- [Quantum-Informed Machine Learning for Predicting Spatiotemporal Chaos with Practical Quantum Advantage](https://mlsanity.com/paper/2507.19861/) (2025)
- [PICT -- A Differentiable, GPU-Accelerated Multi-Block PISO Solver for Simulation-Coupled Learning Tasks in Fluid Dynamics](https://mlsanity.com/paper/2505.16992/) (2025)
- [Neural modal ordinary differential equations: Integrating physics-based modeling with neural ordinary differential equations for modeling high-dimensional monitored structures](https://mlsanity.com/paper/2207.07883/) (2022)
- [Multi-Agent Reinforcement Learning: Methods, Applications, Visionary Prospects, and Challenges](https://mlsanity.com/paper/2305.10091/) (2023)
- [Toward Discretization-Consistent Closure Schemes for Large Eddy Simulation Using Reinforcement Learning](https://mlsanity.com/paper/2309.06260/) (2023)
- [Gradient-free online learning of subgrid-scale dynamics with neural emulators](https://mlsanity.com/paper/2310.19385/) (2023)
- [Data Assimilation in Chaotic Systems Using Deep Reinforcement Learning](https://mlsanity.com/paper/2401.00916/) (2024)
- [Physics-Preserving AI-Accelerated Simulations of Plasma Turbulence](https://mlsanity.com/paper/2309.16400/) (2023)
- [Mean flow data assimilation using physics-constrained Graph Neural Networks](https://mlsanity.com/paper/2411.09476/) (2024)
- [Remember and Forget Experience Replay for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2203.13319/) (2022)
- [Multi-agent reinforcement learning for wall modeling in LES of flow over periodic hills](https://mlsanity.com/paper/2211.16427/) (2022)
- [Predicting the wall-shear stress and wall pressure through convolutional neural networks](https://mlsanity.com/paper/2303.00706/) (2023)
- [Discovering Individual Rewards in Collective Behavior through Inverse Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2305.10548/) (2023)
- [In Situ Framework for Coupling Simulation and Machine Learning with Application to CFD](https://mlsanity.com/paper/2306.12900/) (2023)
- [Uncovering wall-shear stress dynamics from neural-network enhanced fluid flow measurements](https://mlsanity.com/paper/2310.11147/) (2023)
- [Extreme Event Prediction with Multi-agent Reinforcement Learning-based Parametrization of Atmospheric and Oceanic Turbulence](https://mlsanity.com/paper/2312.00907/) (2023)

## Abstract (arXiv)

The predictive capabilities of turbulent flow simulations, critical for aerodynamic design and weather prediction, hinge on the choice of turbulence models. The abundance of data from experiments and simulations and the advent of machine learning have provided a boost to turbulence modeling efforts. However, simulations of turbulent flows remain hindered by the inability of heuristics and supervised learning to model the near-wall dynamics. We address this challenge by introducing scientific multi-agent reinforcement learning (SciMARL) for the discovery of wall models for large-eddy simulations (LES). In SciMARL, discretization points act also as cooperating agents that learn to supply the LES closure model. The agents self-learn using limited data and generalize to extreme Reynolds numbers and previously unseen geometries. The present simulations reduce by several orders of magnitude the computational cost over fully-resolved simulations while reproducing key flow quantities. We believe that SciMARL creates unprecedented capabilities for the simulation of turbulent flows.