# Coupling parameter and particle dynamics for adaptive sampling in Neural Galerkin schemes

arXiv: https://arxiv.org/abs/2306.15630 · HTML: https://mlsanity.com/paper/2306.15630/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 76 references that are in the map.
- First posted on arXiv 27 Jun 2023.

## Builds on (most cited first, 28 of 28)

- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [DeepXDE: A deep learning library for solving differential equations](https://mlsanity.com/paper/1907.04502/) (2019)
- [The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems](https://mlsanity.com/paper/1710.00211/) (2017)
- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016)
- [Recent Advances in Adversarial Training for Adversarial Robustness](https://mlsanity.com/paper/2102.01356/) (2021)
- [A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks](https://mlsanity.com/paper/2207.10289/) (2022)
- [Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations](https://mlsanity.com/paper/1706.04702/) (2017)
- [Solving and Learning Nonlinear PDEs with Gaussian Processes](https://mlsanity.com/paper/2103.12959/) (2021)
- [Solving Allen-Cahn and Cahn-Hilliard Equations using the Adaptive Physics Informed Neural Networks](https://mlsanity.com/paper/2007.04542/) (2020)
- [A unified deep artificial neural network approach to partial differential equations in complex geometries](https://mlsanity.com/paper/1711.06464/) (2017)
- [Measuring Sample Quality with Kernels](https://mlsanity.com/paper/1703.01717/) (2017)
- [A fast and accurate physics-informed neural network reduced order model with shallow masked autoencoder](https://mlsanity.com/paper/2009.11990/) (2020)
- [Maximum Principle Based Algorithms for Deep Learning](https://mlsanity.com/paper/1710.09513/) (2017)
- [Deep Learning via Dynamical Systems: An Approximation Perspective](https://mlsanity.com/paper/1912.10382/) (2019)
- [DAS-PINNs: A deep adaptive sampling method for solving high-dimensional partial differential equations](https://mlsanity.com/paper/2112.14038/) (2021)
- [CROM: Continuous Reduced-Order Modeling of PDEs Using Implicit Neural Representations](https://mlsanity.com/paper/2206.02607/) (2022)
- [Neural Galerkin Schemes with Active Learning for High-Dimensional Evolution Equations](https://mlsanity.com/paper/2203.01360/) (2022)
- [Evolutional Deep Neural Network](https://mlsanity.com/paper/2103.09959/) (2021)
- [Scaling limit of the Stein variational gradient descent: the mean field regime](https://mlsanity.com/paper/1805.04035/) (2018)
- [Solving for high dimensional committor functions using artificial neural networks](https://mlsanity.com/paper/1802.10275/) (2018)
- [Computing Committor Functions for the Study of Rare Events Using Deep Learning](https://mlsanity.com/paper/1906.06285/) (2019)
- [A Deep Learning approach to Reduced Order Modelling of Parameter Dependent Partial Differential Equations](https://mlsanity.com/paper/2103.06183/) (2021)
- [Lagrangian PINNs: A causality-conforming solution to failure modes of physics-informed neural networks](https://mlsanity.com/paper/2205.02902/) (2022)
- [Error Analysis of Kernel/GP Methods for Nonlinear and Parametric PDEs](https://mlsanity.com/paper/2305.04962/) (2023)
- [A Stable and Scalable Method for Solving Initial Value PDEs with Neural Networks](https://mlsanity.com/paper/2304.14994/) (2023)
- [Variational training of neural network approximations of solution maps for physical models](https://mlsanity.com/paper/1905.02789/) (2019)
- [A Novel Adaptive Causal Sampling Method for Physics-Informed Neural Networks](https://mlsanity.com/paper/2210.12914/) (2022)

## Built on by (most cited first, 10 of 10)

- [CoLoRA: Continuous low-rank adaptation for reduced implicit neural modeling of parameterized partial differential equations](https://mlsanity.com/paper/2402.14646/) (2024)
- [Sequential-in-time training of nonlinear parametrizations for solving time-dependent partial differential equations](https://mlsanity.com/paper/2404.01145/) (2024)
- [Deep adaptive sampling for surrogate modeling without labeled data](https://mlsanity.com/paper/2402.11283/) (2024)
- [Nonlinear embeddings for conserving Hamiltonians and other quantities with Neural Galerkin schemes](https://mlsanity.com/paper/2310.07485/) (2023)
- [Mitigating distribution shift in machine learning-augmented hybrid simulation](https://mlsanity.com/paper/2401.09259/) (2024)
- [Multi evolutional deep neural networks (Multi-EDNN)](https://mlsanity.com/paper/2407.12293/) (2024)
- [Filtered Neural Galerkin model reduction schemes for efficient propagation of initial condition uncertainties in digital twins](https://mlsanity.com/paper/2511.00670/) (2025)
- [Nonlinear model reduction for transport-dominated problems](https://mlsanity.com/paper/2602.01397/) (2026)
- [A Dirac-Frenkel-Onsager principle: Instantaneous residual minimization with gauge momentum for nonlinear parametrizations of PDE solutions](https://mlsanity.com/paper/2605.00284/) (2026)
- [A transition-density-based operator learning method for Fokker-Planck equations with various initial conditions](https://mlsanity.com/paper/2606.09434/) (2026)

## Abstract (arXiv)

Training nonlinear parametrizations such as deep neural networks to numerically approximate solutions of partial differential equations is often based on minimizing a loss that includes the residual, which is analytically available in limited settings only. At the same time, empirically estimating the training loss is challenging because residuals and related quantities can have high variance, especially for transport-dominated and high-dimensional problems that exhibit local features such as waves and coherent structures. Thus, estimators based on data samples from un-informed, uniform distributions are inefficient. This work introduces Neural Galerkin schemes that estimate the training loss with data from adaptive distributions, which are empirically represented via ensembles of particles. The ensembles are actively adapted by evolving the particles with dynamics coupled to the nonlinear parametrizations of the solution fields so that the ensembles remain informative for estimating the training loss. Numerical experiments indicate that few dynamic particles are sufficient for obtaining accurate empirical estimates of the training loss, even for problems with local features and with high-dimensional spatial domains.