# PINNs Failure Modes are Overfitting

arXiv: https://arxiv.org/abs/2605.30910 · HTML: https://mlsanity.com/paper/2605.30910/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 46 references that are in the map.
- First posted on arXiv 29 May 2026.

## Builds on (most cited first, 25 of 25)

- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Neural Operator: Graph Kernel Network for Partial Differential Equations](https://mlsanity.com/paper/2003.03485/) (2020)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [When and why PINNs fail to train: A neural tangent kernel perspective](https://mlsanity.com/paper/2007.14527/) (2020)
- [The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems](https://mlsanity.com/paper/1710.00211/) (2017)
- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [Physics-Informed Neural Operator for Learning Partial Differential Equations](https://mlsanity.com/paper/2111.03794/) (2021)
- [On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks](https://mlsanity.com/paper/2012.10047/) (2020)
- [Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems](https://mlsanity.com/paper/2111.02801/) (2021)
- [Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations](https://mlsanity.com/paper/1706.04702/) (2017)
- [Sobolev Training for Neural Networks](https://mlsanity.com/paper/1706.04859/) (2017)
- [Challenges in Training PINNs: A Loss Landscape Perspective](https://mlsanity.com/paper/2402.01868/) (2024)
- [Mitigating Propagation Failures in Physics-informed Neural Networks using Retain-Resample-Release (R3) Sampling](https://mlsanity.com/paper/2207.02338/) (2022)
- [PINNsFormer: A Transformer-Based Framework For Physics-Informed Neural Networks](https://mlsanity.com/paper/2307.11833/) (2023)
- [PINNACLE: PINN Adaptive ColLocation and Experimental points selection](https://mlsanity.com/paper/2404.07662/) (2024)
- [Dual Cone Gradient Descent for Training Physics-Informed Neural Networks](https://mlsanity.com/paper/2409.18426/) (2024)
- [Preconditioning for Physics-Informed Neural Networks](https://mlsanity.com/paper/2402.00531/) (2024)
- [A Unified Hard-Constraint Framework for Solving Geometrically Complex PDEs](https://mlsanity.com/paper/2210.03526/) (2022)
- [FP64 is All You Need: Rethinking Failure Modes in Physics-Informed Neural Networks](https://mlsanity.com/paper/2505.10949/) (2025)
- [Sub-Sequential Physics-Informed Learning with State Space Model](https://mlsanity.com/paper/2502.00318/) (2025)
- [ProPINN: Demystifying Propagation Failures in Physics-Informed Neural Networks](https://mlsanity.com/paper/2502.00803/) (2025)
- [Solving Differential Equations with Constrained Learning](https://mlsanity.com/paper/2410.22796/) (2024)
- [BWLer: Barycentric Weight Layer Elucidates a Precision-Conditioning Tradeoff for PINNs](https://mlsanity.com/paper/2506.23024/) (2025)

## Abstract (arXiv)

Physics-Informed Neural Networks (PINNs) are a common class of machine learning-based partial differential equation (PDE) solvers which train a network to represent a solution by minimizing a residual loss that encodes the PDE. Despite their successes, they are known to fail on certain simple equations, converging to an incorrect solution despite low loss. These failure modes have garnered significant attention in the literature over the past several years, motivating both architectural and optimization based solutions. By directly visualizing the residual, we show that failure modes are the result of overfitting: the loss is minimized on the collocation points, but not elsewhere. Applying regularization causes the failure modes to vanish. Finally, we extend double backpropagation over the full set of residuals, and use it to achieve state-of-the-art performance on four standard failure mode equations with up to 23× fewer collocation points and a vanilla architecture.