# Deep Reinforcement Learning for Turbulence Modeling in Large Eddy Simulations

arXiv: https://arxiv.org/abs/2206.11038 · HTML: https://mlsanity.com/paper/2206.11038/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 46 references that are in the map.
- First posted on arXiv 21 Jun 2022.

## Builds on (most cited first, 14 of 14)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [Machine Learning for Fluid Mechanics](https://mlsanity.com/paper/1905.11075/) (2019)
- [Learning Convolutional Neural Networks for Graphs](https://mlsanity.com/paper/1605.05273/) (2016)
- [Scientific multi-agent reinforcement learning for wall-models of turbulent flows](https://mlsanity.com/paper/2106.11144/) (2021)
- [Deep Neural Networks for Data-Driven Turbulence Models](https://mlsanity.com/paper/1806.04482/) (2018)
- [A Perspective on Machine Learning Methods in Turbulence Modelling](https://mlsanity.com/paper/2010.12226/) (2020)
- [FLEXI: A high order discontinuous Galerkin framework for hyperbolic-parabolic conservation laws](https://mlsanity.com/paper/1910.02858/) (2019)
- [Using Machine Learning at Scale in HPC Simulations with SmartSim: An Application to Ocean Climate Modeling](https://mlsanity.com/paper/2104.09355/) (2021)
- [Deep Reinforcement Learning for Computational Fluid Dynamics on HPC Systems](https://mlsanity.com/paper/2205.06502/) (2022)
- [A machine learning framework for LES closure terms](https://mlsanity.com/paper/2010.03030/) (2020)
- [Deploying deep learning in OpenFOAM with TensorFlow](https://mlsanity.com/paper/2012.00900/) (2020)

## Built on by (most cited first, 16 of 16)

- [Improving aircraft performance using machine learning: a review](https://mlsanity.com/paper/2210.11481/) (2022)
- [Toward Discretization-Consistent Closure Schemes for Large Eddy Simulation Using Reinforcement Learning](https://mlsanity.com/paper/2309.06260/) (2023)
- [Gradient-free online learning of subgrid-scale dynamics with neural emulators](https://mlsanity.com/paper/2310.19385/) (2023)
- [Advanced deep-reinforcement-learning methods for flow control: group-invariant and positional-encoding networks improve learning speed and quality](https://mlsanity.com/paper/2407.17822/) (2024)
- [Gym-preCICE: Reinforcement Learning Environments for Active Flow Control](https://mlsanity.com/paper/2305.02033/) (2023)
- [In Situ Framework for Coupling Simulation and Machine Learning with Application to CFD](https://mlsanity.com/paper/2306.12900/) (2023)
- [Extreme Event Prediction with Multi-agent Reinforcement Learning-based Parametrization of Atmospheric and Oceanic Turbulence](https://mlsanity.com/paper/2312.00907/) (2023)
- [Optimal Lattice Boltzmann Closures through Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2504.14422/) (2025)
- [Combining Machine Learning with Computational Fluid Dynamics using OpenFOAM and SmartSim](https://mlsanity.com/paper/2402.16196/) (2024)
- [Reinforcement learning for anisotropic p-adaptation and error estimation in high-order solvers](https://mlsanity.com/paper/2407.19000/) (2024)
- [Invariant Control Strategies for Active Flow Control using Graph Neural Networks](https://mlsanity.com/paper/2503.22775/) (2025)
- [An Analytical and AI-discovered Stable, Accurate, and Generalizable Subgrid-scale Closure for Geophysical Turbulence](https://mlsanity.com/paper/2509.20365/) (2025)
- [Replacing Tunable Parameters in Weather and Climate Models with State-Dependent Functions using Reinforcement Learning](https://mlsanity.com/paper/2601.04268/) (2026)
- [Parameter-Aware Ensemble SINDy for Interpretable Symbolic SGS Closure](https://mlsanity.com/paper/2508.14085/) (2025)
- [Addressing A Posteriori Performance Degradation in Neural Network Subgrid Stress Models](https://mlsanity.com/paper/2511.17475/) (2025)
- [Spectrally Regularized Latent Flow Matching for Turbulence Generation](https://mlsanity.com/paper/2606.11691/) (2026)

## Abstract (arXiv)

Over the last years, supervised learning (SL) has established itself as the state-of-the-art for data-driven turbulence modeling. In the SL paradigm, models are trained based on a dataset, which is typically computed a priori from a high-fidelity solution by applying the respective filter function, which separates the resolved and the unresolved flow scales. For implicitly filtered large eddy simulation (LES), this approach is infeasible, since here, the employed discretization itself acts as an implicit filter function. As a consequence, the exact filter form is generally not known and thus, the corresponding closure terms cannot be computed even if the full solution is available. The reinforcement learning (RL) paradigm can be used to avoid this inconsistency by training not on a previously obtained training dataset, but instead by interacting directly with the dynamical LES environment itself. This allows to incorporate the potentially complex implicit LES filter into the training process by design. In this work, we apply a reinforcement learning framework to find an optimal eddy-viscosity for implicitly filtered large eddy simulations of forced homogeneous isotropic turbulence. For this, we formulate the task of turbulence modeling as an RL task with a policy network based on convolutional neural networks that adapts the eddy-viscosity in LES dynamically in space and time based on the local flow state only. We demonstrate that the trained models can provide long-term stable simulations and that they outperform established analytical models in terms of accuracy. In addition, the models generalize well to other resolutions and discretizations. We thus demonstrate that RL can provide a framework for consistent, accurate and stable turbulence modeling especially for implicitly filtered LES.