mlsanityData to 14 Oct 2026

ANaGRAM: A Natural Gradient Relative to Adapted Model for efficient PINNs learning

arXiv 2412.10782 ↗First posted 14 Dec 2024cs.LG cs.AI cs.NAcited by 7

Not on the map. Why?

Abstract (arXiv)

In the recent years, Physics Informed Neural Networks (PINNs) have received strong interest as a method to solve PDE driven systems, in particular for data assimilation purpose. This method is still in its infancy, with many shortcomings and failures that remain not properly understood. In this paper we propose a natural gradient approach to PINNs which contributes to speed-up and improve the accuracy of the training. Based on an in depth analysis of the differential geometric structures of the problem, we come up with two distinct contributions: (i) a new natural gradient algorithm that scales as min(P2S, S2P), where P is the number of parameters, and S the batch size; (ii) a mathematically principled reformulation of the PINNs problem that allows the extension of natural gradient to it, with proved connections to Green's function theory.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 25 of its 61 references that are in the map. First posted on arXiv 14 Dec 2024.

Builds on

25 of 61 references.

References · 25

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  2. 2015Automatic differentiation in machine learning: a surveycited by 855
  3. 2015Optimizing Neural Networks with Kronecker-factored Approximate Curvaturecited by 796
  4. 2021Characterizing possible failure modes in physics-informed neural networkscited by 643
  5. 2019DeepXDE: A deep learning library for solving differential equationscited by 619
  6. 2022Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's nextcited by 608
  7. 2020Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanismcited by 233
  8. 2018Fast Approximate Natural Gradient Descent in a Kronecker-factored Eigenbasiscited by 146
  9. 2016A Kronecker-factored approximate Fisher matrix for convolution layerscited by 141
  10. 2018Deep Learning of Vortex Induced Vibrationscited by 76
  11. 2023Can Physics-Informed Neural Networks beat the Finite Element Method?cited by 68
  12. 2019Gram-Gauss-Newton Method: Learning Overparameterized Neural Networks for Regression Problemscited by 40
13 more
  1. 2020Deep learning of free boundary and Stefan problemscited by 35
  2. 2013Riemannian metrics for neural networks I: feedforward networkscited by 30
  3. 2024Gauss-Newton Natural Gradient Descent for Physics-Informed Computational Fluid Dynamicscited by 28
  4. 2023An operator preconditioning perspective on training in physics-informed machine learningcited by 27
  5. 2024Kronecker-Factored Approximate Curvature for Physics-Informed Neural Networkscited by 21
  6. 2024Position: Optimization in SciML Should Employ the Function Space Geometrycited by 20
  7. 2022Competitive Physics Informed Networkscited by 19
  8. 2018Deep Learning of Turbulent Scalar Mixingcited by 17
  9. 2024Preconditioning for Physics-Informed Neural Networkscited by 13
  10. 2024Growing Tiny Networks: Spotting Expressivity Bottlenecks and Fixing Them Optimallycited by 9
  11. 2017True Asymptotic Natural Gradient Optimizationcited by 5
  12. 2024Convergence Analysis of Natural Gradient Descent for Over-parameterized Physics-Informed Neural Networkscited by 5
  13. 2022Generalized Tangent Kernel: A Unified Geometric Foundation for Natural Gradient and Standard Gradientcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 4.

7 papers.

  1. 2025Dual Natural Gradient Descent for Scalable Training of Physics-Informed Neural Networkscited by 6
  2. 2026Learning PDE Solvers with Physics and Data: A Unifying View of Physics-Informed Neural Networks and Neural Operatorscited by 6
  3. 2025A Sketch-and-Project Analysis of Subsampled Natural Gradient Algorithmscited by 2
  4. 2026Natural gradient descent with momentumcited by 2
  5. 2025Distribution-Aware Tensor Decomposition for Compression of Convolutional Neural Networkscited by 0
  6. 2026SS-ESOAP: Self-Scaled Adaptive Preconditioning for Physics-Informed Learningcited by 0
  7. 2026Beyond PINNs: A Unified Gauss--Newton and Petrov--Galerkin Framework for Neural and Hybrid PDE Solverscited by 0