# Beyond PINNs: A Unified Gauss--Newton and Petrov--Galerkin Framework for Neural and Hybrid PDE Solvers

arXiv: https://arxiv.org/abs/2609.20641 · HTML: https://mlsanity.com/paper/2609.20641/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 46 references that are in the map.
- First posted on arXiv 17 Sep 2026.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [hp-VPINNs: Variational Physics-Informed Neural Networks With Domain Decomposition](https://mlsanity.com/paper/2003.05385/) (2020)
- [A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks](https://mlsanity.com/paper/2207.10289/) (2022)
- [Variational Physics-Informed Neural Networks For Solving Partial Differential Equations](https://mlsanity.com/paper/1912.00873/) (2019)
- [Efficient training of physics-informed neural networks via importance sampling](https://mlsanity.com/paper/2104.12325/) (2021)
- [Mitigating Propagation Failures in Physics-informed Neural Networks using Retain-Resample-Release (R3) Sampling](https://mlsanity.com/paper/2207.02338/) (2022)
- [Can Physics-Informed Neural Networks beat the Finite Element Method?](https://mlsanity.com/paper/2302.04107/) (2023)
- [PINNACLE: PINN Adaptive ColLocation and Experimental points selection](https://mlsanity.com/paper/2404.07662/) (2024)
- [Gauss-Newton Natural Gradient Descent for Physics-Informed Computational Fluid Dynamics](https://mlsanity.com/paper/2402.10680/) (2024)
- [Curvature-Aware Optimization for High-Accuracy Physics-Informed Neural Networks](https://mlsanity.com/paper/2604.05230/) (2026)
- [ANaGRAM: A Natural Gradient Relative to Adapted Model for efficient PINNs learning](https://mlsanity.com/paper/2412.10782/) (2024)
- [Dual Natural Gradient Descent for Scalable Training of Physics-Informed Neural Networks](https://mlsanity.com/paper/2505.21404/) (2025)
- [Fixed-budget online adaptive learning for physics-informed neural networks. Towards parameterized problem inference](https://mlsanity.com/paper/2212.11776/) (2022)
- [Natural gradient descent with momentum](https://mlsanity.com/paper/2604.15554/) (2026)
- [AMStraMGRAM: Adaptive Multi-cutoff Strategy Modification for ANaGRAM](https://mlsanity.com/paper/2510.15998/) (2025)
- [A new initialisation to Control Gradients in Sinusoidal Neural network](https://mlsanity.com/paper/2512.06427/) (2025)
- [Error whitening: Why Gauss-Newton outperforms Newton](https://mlsanity.com/paper/2605.11316/) (2026)
- [An Optimisation Framework for the Well-Conditioned Training of Physics-Informed Neural Networks](https://mlsanity.com/paper/2607.02194/) (2026)

## Abstract (arXiv)

Physics-informed neural networks and finite element methods provide two different paradigms for the numerical approximation of partial differential equations: the former are commonly trained by minimizing pointwise strong residuals, whereas the latter are naturally built from weak variational formulations and the finite-dimensional systems obtained after discretization. In this work, we introduce a common framework based on the discretization of functional Gauss--Newton problems by finite families of linear measurements. We show that, through an appropriate duality pairing, the linear measurements can be represented by test functions. The resulting Gauss--Newton system is then precisely a Petrov--Galerkin discretization of the linearized functional problem. This perspective recovers pointwise collocation and natural-gradient constructions as particular cases, while making the choice of test functions an explicit algorithmic design choice. We specialize this framework to elliptic problems, where it naturally leads to weak residual formulations and to a hybrid finite element--neural construction acting on complementary approximation spaces. Numerical experiments support the proposed framework and demonstrate the effectiveness of weak Gauss--Newton formulations and hybrid finite element--neural approximations.