# Neural Galerkin Schemes with Active Learning for High-Dimensional Evolution Equations

arXiv: https://arxiv.org/abs/2203.01360 · HTML: https://mlsanity.com/paper/2203.01360/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 50 references that are in the map.
- First posted on arXiv 2 Mar 2022.

## Builds on (most cited first, 16 of 16)

- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [Machine learning accelerated computational fluid dynamics](https://mlsanity.com/paper/2102.01010/) (2021)
- [Neural Operator: Learning Maps Between Function Spaces](https://mlsanity.com/paper/2108.08481/) (2021)
- [Model Reduction and Neural Networks for Parametric PDEs](https://mlsanity.com/paper/2005.03180/) (2020)
- [Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations](https://mlsanity.com/paper/1706.04702/) (2017)
- [A unified deep artificial neural network approach to partial differential equations in complex geometries](https://mlsanity.com/paper/1711.06464/) (2017)
- [Lift & Learn: Physics-informed machine learning for large-scale nonlinear dynamical systems](https://mlsanity.com/paper/1912.08177/) (2019)
- [A fast and accurate physics-informed neural network reduced order model with shallow masked autoencoder](https://mlsanity.com/paper/2009.11990/) (2020)
- [Evolutional Deep Neural Network](https://mlsanity.com/paper/2103.09959/) (2021)
- [Derivative-Informed Projected Neural Networks for High-Dimensional Parametric Maps Governed by PDEs](https://mlsanity.com/paper/2011.15110/) (2020)
- [Solving for high dimensional committor functions using artificial neural networks](https://mlsanity.com/paper/1802.10275/) (2018)
- [Computing Committor Functions for the Study of Rare Events Using Deep Learning](https://mlsanity.com/paper/1906.06285/) (2019)
- [Global convergence of neuron birth-death dynamics](https://mlsanity.com/paper/1902.01843/) (2019)
- [Some observations on high-dimensional partial differential equations with Barron data](https://mlsanity.com/paper/2012.01484/) (2020)
- [Non-linear manifold ROM with Convolutional Autoencoders and Reduced Over-Collocation method](https://mlsanity.com/paper/2203.00360/) (2022)

## Built on by (most cited first, 30 of 56)

- [Multiple Physics Pretraining for Physical Surrogate Models](https://mlsanity.com/paper/2310.02994/) (2023)
- [Probability flow solution of the Fokker-Planck equation](https://mlsanity.com/paper/2206.04642/) (2022)
- [Neural Implicit Flow: a mesh-agnostic dimensionality reduction paradigm of spatio-temporal data](https://mlsanity.com/paper/2204.03216/) (2022)
- [Walrus: A Cross-Domain Foundation Model for Continuum Dynamics](https://mlsanity.com/paper/2511.15684/) (2025)
- [Implicit Neural Spatial Representations for Time-dependent PDEs](https://mlsanity.com/paper/2210.00124/) (2022)
- [PDE-Transformer: Efficient and Versatile Transformers for Physics Simulations](https://mlsanity.com/paper/2505.24717/) (2025)
- [An operator preconditioning perspective on training in physics-informed machine learning](https://mlsanity.com/paper/2310.05801/) (2023)
- [UPS: Efficiently Building Foundation Models for PDE Solving via Cross-Modal Adaptation](https://mlsanity.com/paper/2403.07187/) (2024)
- [Active Learning for Neural PDE Solvers](https://mlsanity.com/paper/2408.01536/) (2024)
- [FMint: Bridging Human Designed and Data Pretrained Models for Differential Equation Foundation Model](https://mlsanity.com/paper/2404.14688/) (2024)
- [Randomized Sparse Neural Galerkin Schemes for Solving Evolution Equations with Deep Networks](https://mlsanity.com/paper/2310.04867/) (2023)
- [A Stable and Scalable Method for Solving Initial Value PDEs with Neural Networks](https://mlsanity.com/paper/2304.14994/) (2023)
- [Evolve Smoothly, Fit Consistently: Learning Smooth Latent Dynamics For Advection-Dominated Systems](https://mlsanity.com/paper/2301.10391/) (2023)
- [Neural Spectral Methods: Self-supervised learning in the spectral domain](https://mlsanity.com/paper/2312.05225/) (2023)
- [CoLoRA: Continuous low-rank adaptation for reduced implicit neural modeling of parameterized partial differential equations](https://mlsanity.com/paper/2402.14646/) (2024)
- [A Multifidelity deep operator network approach to closure for multiscale systems](https://mlsanity.com/paper/2303.08893/) (2023)
- [Coupling parameter and particle dynamics for adaptive sampling in Neural Galerkin schemes](https://mlsanity.com/paper/2306.15630/) (2023)
- [Parametric model reduction of mean-field and stochastic systems via higher-order action matching](https://mlsanity.com/paper/2410.12000/) (2024)
- [Sequential-in-time training of nonlinear parametrizations for solving time-dependent partial differential equations](https://mlsanity.com/paper/2404.01145/) (2024)
- [Scientific Machine Learning Based Reduced-Order Models for Plasma Turbulence Simulations](https://mlsanity.com/paper/2401.05972/) (2024)
- [Deep Learning for Continuous-Time Stochastic Control with Jumps](https://mlsanity.com/paper/2505.15602/) (2025)
- [Neural Control of Parametric Solutions for High-dimensional Evolution PDEs](https://mlsanity.com/paper/2302.00045/) (2023)
- [Operator learning for hyperbolic partial differential equations](https://mlsanity.com/paper/2312.17489/) (2023)
- [Latent Dynamics Networks (LDNets): learning the intrinsic dynamics of spatio-temporal processes](https://mlsanity.com/paper/2305.00094/) (2023)
- [Weak Generative Sampler to Efficiently Sample Invariant Distribution of Stochastic Differential Equation](https://mlsanity.com/paper/2405.19256/) (2024)
- [Deep Stochastic Mechanics](https://mlsanity.com/paper/2305.19685/) (2023)
- [Nonlinear embeddings for conserving Hamiltonians and other quantities with Neural Galerkin schemes](https://mlsanity.com/paper/2310.07485/) (2023)
- [Embed and Emulate: Contrastive representations for simulation-based inference](https://mlsanity.com/paper/2409.18402/) (2024)
- [A hybrid FEM-PINN method for time-dependent partial differential equations](https://mlsanity.com/paper/2409.02810/) (2024)
- [A Natural Primal-Dual Hybrid Gradient Method for Adversarial Neural Network Training on Solving Partial Differential Equations](https://mlsanity.com/paper/2411.06278/) (2024)

## Abstract (arXiv)

Deep neural networks have been shown to provide accurate function approximations in high dimensions. However, fitting network parameters requires informative training data that are often challenging to collect in science and engineering applications. This work proposes Neural Galerkin schemes based on deep learning that generate training data with active learning for numerically solving high-dimensional partial differential equations. Neural Galerkin schemes build on the Dirac-Frenkel variational principle to train networks by minimizing the residual sequentially over time, which enables adaptively collecting new training data in a self-informed manner that is guided by the dynamics described by the partial differential equations. This is in contrast to other machine learning methods that aim to fit network parameters globally in time without taking into account training data acquisition. Our finding is that the active form of gathering training data of the proposed Neural Galerkin schemes is key for numerically realizing the expressive power of networks in high dimensions. Numerical experiments demonstrate that Neural Galerkin schemes have the potential to enable simulating phenomena and processes with many variables for which traditional and other deep-learning-based solvers fail, especially when features of the solutions evolve locally such as in high-dimensional wave propagation problems and interacting particle systems described by Fokker-Planck and kinetic equations.