# Gauss-Newton Natural Gradient Descent for Physics-Informed Computational Fluid Dynamics

arXiv: https://arxiv.org/abs/2402.10680 · HTML: https://mlsanity.com/paper/2402.10680/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Feb 2024.

## Built on by (most cited first, 28 of 28)

- [From PINNs to PIKANs: Recent Advances in Physics-Informed Machine Learning](https://mlsanity.com/paper/2410.13228/) (2024)
- [Gradient Alignment in Physics-informed Neural Networks: A Second-Order Optimization Perspective](https://mlsanity.com/paper/2502.00604/) (2025)
- [Optimizing the Optimizer for Physics-Informed Neural Networks and Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2501.16371/) (2025)
- [Kronecker-Factored Approximate Curvature for Physics-Informed Neural Networks](https://mlsanity.com/paper/2405.15603/) (2024)
- [Dynamical Measure Transport and Neural PDE Solvers for Sampling](https://mlsanity.com/paper/2407.07873/) (2024)
- [Improving Energy Natural Gradient Descent through Woodbury, Momentum, and Randomization](https://mlsanity.com/paper/2505.12149/) (2025)
- [Curvature-Aware Optimization for High-Accuracy Physics-Informed Neural Networks](https://mlsanity.com/paper/2604.05230/) (2026)
- [ANaGRAM: A Natural Gradient Relative to Adapted Model for efficient PINNs learning](https://mlsanity.com/paper/2412.10782/) (2024)
- [High precision PINNs in unbounded domains: application to singularity formulation in PDEs](https://mlsanity.com/paper/2506.19243/) (2025)
- [Operator Learning at Machine Precision](https://mlsanity.com/paper/2511.19980/) (2025)
- [A Variational Framework for Residual-Based Adaptivity in Neural PDE Solvers and Operator Learning](https://mlsanity.com/paper/2509.14198/) (2025)
- [A Structure-Guided Gauss-Newton Method for Shallow ReLU Neural Network](https://mlsanity.com/paper/2404.05064/) (2024)
- [Physics-constrained DeepONet for Surrogate CFD models: a curved backward-facing step case](https://mlsanity.com/paper/2503.11196/) (2025)
- [Enforcing hidden physics in physics-informed neural networks](https://mlsanity.com/paper/2511.14348/) (2025)
- [Riemann Tensor Neural Networks: Learning Conservative Systems with Physics-Constrained Networks](https://mlsanity.com/paper/2503.00755/) (2025)
- [Natural gradient descent with momentum](https://mlsanity.com/paper/2604.15554/) (2026)
- [Diverse Topology Optimization using Modulated Neural Fields](https://mlsanity.com/paper/2502.13174/) (2025)
- [Accelerating Natural Gradient Descent for PINNs with Randomized Numerical Linear Algebra](https://mlsanity.com/paper/2505.11638/) (2025)
- [On the Convergence Behavior of Preconditioned Gradient Descent Toward the Rich Learning Regime](https://mlsanity.com/paper/2601.03162/) (2026)
- [Error whitening: Why Gauss-Newton outperforms Newton](https://mlsanity.com/paper/2605.11316/) (2026)
- [An Optimisation Framework for the Well-Conditioned Training of Physics-Informed Neural Networks](https://mlsanity.com/paper/2607.02194/) (2026)
- [Gauss-Newton Natural Gradient Descent for Shape Learning](https://mlsanity.com/paper/2602.00099/) (2026)
- [Singularity Formation: Synergy in Theoretical, Numerical and Machine Learning Approaches](https://mlsanity.com/paper/2604.16842/) (2026)
- [Convergence Analysis of Newton's Method for Neural Networks in the Overparameterized Limit](https://mlsanity.com/paper/2605.08352/) (2026)
- [Energy Manifold Natural Gradient Descent: Riemannian Optimization for Neural PDE Solvers](https://mlsanity.com/paper/2607.22004/) (2026)
- [From Non-Convex Self-Concordant Regularization to Scalable Quasi-Newton Training of PINNs](https://mlsanity.com/paper/2608.04206/) (2026)
- [SS-ESOAP: Self-Scaled Adaptive Preconditioning for Physics-Informed Learning](https://mlsanity.com/paper/2608.29448/) (2026)
- [Beyond PINNs: A Unified Gauss--Newton and Petrov--Galerkin Framework for Neural and Hybrid PDE Solvers](https://mlsanity.com/paper/2609.20641/) (2026)

## Abstract (arXiv)

We propose Gauss-Newton's method in function space for the solution of the Navier-Stokes equations in the physics-informed neural network (PINN) framework. Upon discretization, this yields a natural gradient method that provably mimics the function space dynamics. Our computational results demonstrate close to single-precision accuracy measured in relative L2 norm on a number of benchmark problems. To the best of our knowledge, this constitutes the first contribution in the PINN literature that solves the Navier-Stokes equations to this degree of accuracy. Finally, we show that given a suitable integral discretization, the proposed optimization algorithm agrees with Gauss-Newton's method in parameter space. This allows a matrix-free formulation enabling efficient scalability to large network sizes.