# DPM: A deep learning PDE augmentation method (with application to large-eddy simulation)

arXiv: https://arxiv.org/abs/1911.09145 · HTML: https://mlsanity.com/paper/1911.09145/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Nov 2019.

## Built on by (most cited first, 30 of 41)

- [Machine learning accelerated computational fluid dynamics](https://mlsanity.com/paper/2102.01010/) (2021)
- [Learned Coarse Models for Efficient Turbulence Simulation](https://mlsanity.com/paper/2112.15275/) (2021)
- [Benchmarking Autoregressive Conditional Diffusion Models for Turbulent Flow Simulation](https://mlsanity.com/paper/2309.01745/) (2023)
- [Learned Turbulence Modelling with Differentiable Fluid Solvers: Physics-based Loss-functions and Optimisation Horizons](https://mlsanity.com/paper/2202.06988/) (2022)
- [JAX-FLUIDS: A fully-differentiable high-order computational fluid dynamics solver for compressible two-phase flows](https://mlsanity.com/paper/2203.13760/) (2022)
- [A posteriori learning for quasi-geostrophic turbulence parametrization](https://mlsanity.com/paper/2204.03911/) (2022)
- [Data-Efficient Operator Learning via Unsupervised Pretraining and In-Context Learning](https://mlsanity.com/paper/2402.15734/) (2024)
- [Data-driven Control of Agent-based Models: an Equation/Variable-free Machine Learning Approach](https://mlsanity.com/paper/2207.05779/) (2022)
- [Embedded training of neural-network sub-grid-scale turbulence models](https://mlsanity.com/paper/2105.01030/) (2021)
- [PDE-constrained Models with Neural Network Terms: Optimization and Global Convergence](https://mlsanity.com/paper/2105.08633/) (2021)
- [Multiscale Neural Operator: Learning Fast and Grid-independent PDE Solvers](https://mlsanity.com/paper/2207.11417/) (2022)
- [Deep Learning Closure Models for Large-Eddy Simulation of Flows around Bluff Bodies](https://mlsanity.com/paper/2208.03498/) (2022)
- [Generalized Neural Closure Models with Interpretability](https://mlsanity.com/paper/2301.06198/) (2023)
- [JAX-Fluids 2.0: Towards HPC for Differentiable CFD of Compressible Two-phase Flows](https://mlsanity.com/paper/2402.05193/) (2024)
- [AutoKE: An automatic knowledge embedding framework for scientific machine learning](https://mlsanity.com/paper/2205.05390/) (2022)
- [Comparison of neural closure models for discretised PDEs](https://mlsanity.com/paper/2210.14675/) (2022)
- [PICT -- A Differentiable, GPU-Accelerated Multi-Block PISO Solver for Simulation-Coupled Learning Tasks in Fluid Dynamics](https://mlsanity.com/paper/2505.16992/) (2025)
- [Gradient-free online learning of subgrid-scale dynamics with neural emulators](https://mlsanity.com/paper/2310.19385/) (2023)
- [Dynamic Deep Learning LES Closures: Online Optimization With Embedded DNS](https://mlsanity.com/paper/2303.02338/) (2023)
- [INC: An Indirect Neural Corrector for Auto-Regressive Hybrid PDE Solvers](https://mlsanity.com/paper/2511.12764/) (2025)
- [Probabilistic partition of unity networks for high-dimensional regression problems](https://mlsanity.com/paper/2210.02694/) (2022)
- [A TVD neural network closure and application to turbulent combustion](https://mlsanity.com/paper/2408.03413/) (2024)
- [Learning Transport Processes with Machine Intelligence](https://mlsanity.com/paper/2109.13096/) (2021)
- [Exploring Physical Latent Spaces for High-Resolution Flow Restoration](https://mlsanity.com/paper/2211.11298/) (2022)
- [Differentiable Turbulence: Closure as a partial differential equation constrained optimization](https://mlsanity.com/paper/2307.03683/) (2023)
- [NORi: An ML-Augmented Ocean Boundary Layer Parameterization](https://mlsanity.com/paper/2512.04452/) (2025)
- [Enhancing generalization in endwall film cooling prediction: Incorporating the superposition principle into transformer-based neural operators](https://mlsanity.com/paper/2609.31633/) (2026)
- [SPDCinv: Inverse Quantum-Optical Design of High-Dimensional Qudits](https://mlsanity.com/paper/2112.05934/) (2021)
- [A Spectral Approach for Learning Spatiotemporal Neural Differential Equations](https://mlsanity.com/paper/2309.16131/) (2023)
- [Learning-based Multi-continuum Model for Multiscale Flow Problems](https://mlsanity.com/paper/2403.14084/) (2024)

## Abstract (arXiv)

Machine learning for scientific applications faces the challenge of limited data. We propose a framework that leverages a priori known physics to reduce overfitting when training on relatively small datasets. A deep neural network is embedded in a partial differential equation (PDE) that expresses the known physics and learns to describe the corresponding unknown or unrepresented physics from the data. Crafted as such, the neural network can also provide corrections for erroneously represented physics, such as discretization errors associated with the PDE's numerical solution. Once trained, the deep learning PDE model (DPM) can make out-of-sample predictions for new physical parameters, geometries, and boundary conditions. Our approach optimizes over the functional form of the PDE. Estimating the embedded neural network requires optimizing over the entire PDE, which itself is a function of the neural network. Adjoint partial differential equations are used to efficiently calculate the high-dimensional gradient of the objective function with respect to the neural network parameters. A stochastic adjoint method (SAM), similar in spirit to stochastic gradient descent, further accelerates training. The approach is demonstrated and evaluated for turbulence predictions using large-eddy simulation (LES), a filtered version of the Navier--Stokes equation containing unclosed sub-filter-scale terms. The DPM outperforms the widely-used constant-coefficient and dynamic Smagorinsky models, even for filter sizes so large that these established models become qualitatively incorrect. It also significantly outperforms a priori trained models, which do not account for the full PDE. A relaxation of the discrete enforcement of the divergence-free constraint is also considered, instead allowing the DPM to approximately enforce incompressibility physics.