mlsanityData to 7 Oct 2026

TENG: Time-Evolving Natural Gradient for Solving PDEs With Deep Neural Nets Toward Machine Precision

arXiv 2404.10771 ↗First posted 16 Apr 2024cs.LG cs.NA math.NAcited by 3

Not on the map. Why?

Abstract (arXiv)

Partial differential equations (PDEs) are instrumental for modeling dynamical systems in science and engineering. The advent of neural networks has initiated a significant shift in tackling these complexities though challenges in accuracy persist, especially for initial value problems. In this paper, we introduce the textitTime-Evolving Natural Gradient (TENG), generalizing time-dependent variational principles and optimization-based time integration, leveraging natural gradient optimization to obtain high accuracy in neural-network-based PDE solutions. Our comprehensive development includes algorithms like TENG-Euler and its high-order variants, such as TENG-Heun, tailored for enhanced precision and efficiency. TENG's effectiveness is further validated through its performance, surpassing current leading methods and achieving textitmachine precision in step-by-step optimizations across a spectrum of PDEs, including the heat equation, Allen-Cahn equation, and Burgers' equation.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 22 of its 39 references that are in the map. First posted on arXiv 16 Apr 2024.

Builds on

22 of 39 references.

References · 22

  1. 2018Neural Ordinary Differential Equationscited by 3,733
  2. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  3. 2020Learning to Simulate Complex Physics with Graph Networkscited by 867
  4. 2020Learning Mesh-Based Simulation with Graph Networkscited by 840
  5. 2017DGM: A deep learning algorithm for solving partial differential equationscited by 674
  6. 2017The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problemscited by 581
  7. 2017Solving high-dimensional partial differential equations using deep learningcited by 549
  8. 2021Learning the solution operator of parametric partial differential equations with physics-informed DeepOnetscited by 537
  9. 2017PDE-Net: Learning PDEs from Datacited by 304
  10. 2017Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equationscited by 188
  11. 2013Revisiting Natural Gradient for Deep Networkscited by 157
  12. 2019Quantum Natural Gradientcited by 104
10 more
  1. 2021Long-time integration of parametric evolution equations with physics-informed DeepONetscited by 87
  2. 2020Algorithms for Solving High Dimensional PDEs: From Nonlinear Monte Carlo to Machine Learningcited by 77
  3. 2019Fast Convergence of Natural Gradient Descent for Overparameterized Neural Networkscited by 76
  4. 2021Evolutional Deep Neural Networkcited by 48
  5. 2022Implicit Neural Spatial Representations for Time-dependent PDEscited by 35
  6. 2023Randomized Sparse Neural Galerkin Schemes for Solving Evolution Equations with Deep Networkscited by 18
  7. 2021Gauge Invariant and Anyonic Symmetric Transformer and RNN Quantum States for Quantum Lattice Modelscited by 17
  8. 2022Simulating 2+1D Lattice Quantum Electrodynamics at Finite Density with Neural Flow Wavefunctionscited by 10
  9. 2021Spacetime Neural Network for High Dimensional Quantum Dynamicscited by 9
  10. 2023ANTN: Bridging Autoregressive Neural Networks and Tensor Networks for Quantum Many-Body Simulationcited by 8

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 1.

3 papers.

  1. 2024Kronecker-Factored Approximate Curvature for Physics-Informed Neural Networkscited by 21
  2. 2024Metamizer: a versatile neural optimizer for fast and accurate physics simulationscited by 7
  3. 2025Guiding Time-Varying Generative Models with Natural Gradients on Exponential Family Manifoldcited by 0