# Embedded training of neural-network sub-grid-scale turbulence models

arXiv: https://arxiv.org/abs/2105.01030 · HTML: https://mlsanity.com/paper/2105.01030/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 39 references that are in the map.
- First posted on arXiv 3 May 2021.

## Builds on (most cited first, 6 of 6)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [Universal Differential Equations for Scientific Machine Learning](https://mlsanity.com/paper/2001.04385/) (2020)
- [DPM: A deep learning PDE augmentation method (with application to large-eddy simulation)](https://mlsanity.com/paper/1911.09145/) (2019)
- [Deep Neural Networks for Data-Driven Turbulence Models](https://mlsanity.com/paper/1806.04482/) (2018)
- [Efficient Sampling from Time-Varying Log-Concave Distributions](https://mlsanity.com/paper/1309.5977/) (2013)

## Built on by (most cited first, 13 of 13)

- [Learned Turbulence Modelling with Differentiable Fluid Solvers: Physics-based Loss-functions and Optimisation Horizons](https://mlsanity.com/paper/2202.06988/) (2022)
- [A posteriori learning for quasi-geostrophic turbulence parametrization](https://mlsanity.com/paper/2204.03911/) (2022)
- [Neural Ideal Large Eddy Simulation: Modeling Turbulence with Neural Stochastic Differential Equations](https://mlsanity.com/paper/2306.01174/) (2023)
- [PDE-constrained Models with Neural Network Terms: Optimization and Global Convergence](https://mlsanity.com/paper/2105.08633/) (2021)
- [Deep Learning Closure Models for Large-Eddy Simulation of Flows around Bluff Bodies](https://mlsanity.com/paper/2208.03498/) (2022)
- [Comparison of neural closure models for discretised PDEs](https://mlsanity.com/paper/2210.14675/) (2022)
- [Gradient-free online learning of subgrid-scale dynamics with neural emulators](https://mlsanity.com/paper/2310.19385/) (2023)
- [Dynamic Deep Learning LES Closures: Online Optimization With Embedded DNS](https://mlsanity.com/paper/2303.02338/) (2023)
- [A TVD neural network closure and application to turbulent combustion](https://mlsanity.com/paper/2408.03413/) (2024)
- [OGF: An Online Gradient Flow Method for Optimizing the Statistical Steady-State Time Averages of Unsteady Turbulent Flows](https://mlsanity.com/paper/2507.05149/) (2025)
- [Physics-Based Machine Learning Closures and Wall Models for Hypersonic Transition-Continuum Boundary Layer Predictions](https://mlsanity.com/paper/2507.08986/) (2025)
- [Active Control of Turbulent Airfoil Flows Using Adjoint-based Deep Learning](https://mlsanity.com/paper/2510.07106/) (2025)
- [Flux-form spatiotemporal neural operators for coarse-grained dynamics of multiscale PDEs](https://mlsanity.com/paper/2608.18148/) (2026)

## Abstract (arXiv)

The weights of a deep neural network model are optimized in conjunction with the governing flow equations to provide a model for sub-grid-scale stresses in a temporally developing plane turbulent jet at Reynolds number Re0=6,000. The objective function for training is first based on the instantaneous filtered velocity fields from a corresponding direct numerical simulation, and the training is by a stochastic gradient descent method, which uses the adjoint Navier--Stokes equations to provide the end-to-end sensitivities of the model weights to the velocity fields. In-sample and out-of-sample testing on multiple dual-jet configurations show that its required mesh density in each coordinate direction for prediction of mean flow, Reynolds stresses, and spectra is half that needed by the dynamic Smagorinsky model for comparable accuracy. The same neural-network model trained directly to match filtered sub-grid-scale stresses -- without the constraint of being embedded within the flow equations during the training -- fails to provide a qualitatively correct prediction. The coupled formulation is generalized to train based only on mean-flow and Reynolds stresses, which are more readily available in experiments. The mean-flow training provides a robust model, which is important, though a somewhat less accurate prediction for the same coarse meshes, as might be anticipated due to the reduced information available for training in this case. The anticipated advantage of the formulation is that the inclusion of resolved physics in the training increases its capacity to extrapolate. This is assessed for the case of passive scalar transport, for which it outperforms established models due to improved mixing predictions.