# Comparison of neural closure models for discretised PDEs

arXiv: https://arxiv.org/abs/2210.14675 · HTML: https://mlsanity.com/paper/2210.14675/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 39 references that are in the map.
- First posted on arXiv 26 Oct 2022.

## Builds on (most cited first, 12 of 12)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [How to train your neural ODE: the world of Jacobian and kinetic regularization](https://mlsanity.com/paper/2002.02798/) (2020)
- [Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solvers](https://mlsanity.com/paper/2007.00016/) (2020)
- [Dissecting Neural ODEs](https://mlsanity.com/paper/2002.08071/) (2020)
- [Hybrid Forecasting of Chaotic Processes: Using Machine Learning in Conjunction with a Knowledge-Based Model](https://mlsanity.com/paper/1803.04779/) (2018)
- [DPM: A deep learning PDE augmentation method (with application to large-eddy simulation)](https://mlsanity.com/paper/1911.09145/) (2019)
- [Discretize-Optimize vs. Optimize-Discretize for Time-Series Regression and Continuous Normalizing Flows](https://mlsanity.com/paper/2005.13420/) (2020)
- [Deep Neural Networks for Data-Driven Turbulence Models](https://mlsanity.com/paper/1806.04482/) (2018)
- [Neural Closure Models for Dynamical Systems](https://mlsanity.com/paper/2012.13869/) (2020)
- [Long-time predictive modeling of nonlinear dynamical systems using neural networks](https://mlsanity.com/paper/1805.12547/) (2018)
- [Embedded training of neural-network sub-grid-scale turbulence models](https://mlsanity.com/paper/2105.01030/) (2021)

## Built on by (most cited first, 6 of 6)

- [Benchmarking Autoregressive Conditional Diffusion Models for Turbulent Flow Simulation](https://mlsanity.com/paper/2309.01745/) (2023)
- [Data-Driven Stochastic Closure Modeling via Conditional Diffusion Model and Neural Operator](https://mlsanity.com/paper/2408.02965/) (2024)
- [A Simultaneous Approach for Training Neural Differential-Algebraic Systems of Equations](https://mlsanity.com/paper/2504.04665/) (2025)
- [Trajectory learning for ensemble forecasts via the continuous ranked probability score: a Lorenz '96 case study](https://mlsanity.com/paper/2508.21664/) (2025)
- [LaGSplat: Inferring Physics-Governed Interactive Simulation from Monocular Video Using Latent Lagrangian Gaussian Splatting](https://mlsanity.com/paper/2608.16324/) (2026)
- [Flux-form spatiotemporal neural operators for coarse-grained dynamics of multiscale PDEs](https://mlsanity.com/paper/2608.18148/) (2026)

## Abstract (arXiv)

Neural closure models have recently been proposed as a method for efficiently approximating small scales in multiscale systems with neural networks. The choice of loss function and associated training procedure has a large effect on the accuracy and stability of the resulting neural closure model. In this work, we systematically compare three distinct procedures: "derivative fitting", "trajectory fitting" with discretise-then-optimise, and "trajectory fitting" with optimise-then-discretise. Derivative fitting is conceptually the simplest and computationally the most efficient approach and is found to perform reasonably well on one of the test problems (Kuramoto-Sivashinsky) but poorly on the other (Burgers). Trajectory fitting is computationally more expensive but is more robust and is therefore the preferred approach. Of the two trajectory fitting procedures, the discretise-then-optimise approach produces more accurate models than the optimise-then-discretise approach. While the optimise-then-discretise approach can still produce accurate models, care must be taken in choosing the length of the trajectories used for training, in order to train the models on long-term behaviour while still producing reasonably accurate gradients during training. Two existing theorems are interpreted in a novel way that gives insight into the long-term accuracy of a neural closure model based on how accurate it is in the short term.