# Critical Investigation of Failure Modes in Physics-informed Neural Networks

arXiv: https://arxiv.org/abs/2206.09961 · HTML: https://mlsanity.com/paper/2206.09961/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 27 references that are in the map.
- First posted on arXiv 20 Jun 2022.

## Builds on (most cited first, 9 of 9)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [Deep Learning in Neural Networks: An Overview](https://mlsanity.com/paper/1404.7828/) (2014)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems](https://mlsanity.com/paper/1710.00211/) (2017)
- [Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1711.10566/) (2017)

## Built on by (most cited first, 15 of 15)

- [Investigating and Mitigating Failure Modes in Physics-informed Neural Networks (PINNs)](https://mlsanity.com/paper/2209.09988/) (2022)
- [An adaptive augmented Lagrangian method for training physics and equality constrained artificial neural networks](https://mlsanity.com/paper/2306.04904/) (2023)
- [FP64 is All You Need: Rethinking Failure Modes in Physics-Informed Neural Networks](https://mlsanity.com/paper/2505.10949/) (2025)
- [Approximating Numerical Fluxes Using Fourier Neural Operators for Hyperbolic Conservation Laws](https://mlsanity.com/paper/2401.01783/) (2024)
- [PARCv2: Physics-aware Recurrent Convolutional Neural Networks for Spatiotemporal Dynamics Modeling](https://mlsanity.com/paper/2402.12503/) (2024)
- [Neural Tangent Kernel Analysis to Probe Convergence in Physics-informed Neural Solvers: PIKANs vs. PINNs](https://mlsanity.com/paper/2506.07958/) (2025)
- [ParticleWNN: a Novel Neural Networks Framework for Solving Partial Differential Equations](https://mlsanity.com/paper/2305.12433/) (2023)
- [Beyond Derivative Pathology of PINNs: Variable Splitting Strategy with Convergence Analysis](https://mlsanity.com/paper/2409.20383/) (2024)
- [DGNN: A Neural PDE Solver Induced by Discontinuous Galerkin Methods](https://mlsanity.com/paper/2503.10021/) (2025)
- [Conditionally adaptive augmented Lagrangian method for physics-informed learning of forward and inverse problems](https://mlsanity.com/paper/2508.15695/) (2025)
- [Robust Neural IDA-PBC: passivity-based stabilization under approximations](https://mlsanity.com/paper/2409.16008/) (2024)
- [Architectural Strategies for the optimization of Physics-Informed Neural Networks](https://mlsanity.com/paper/2402.02711/) (2024)
- [Gaussian Variational Schemes on Bounded and Unbounded Domains](https://mlsanity.com/paper/2410.06219/) (2024)
- [IDOL: Meeting Diverse Distribution Shifts with Prior Physics for Tropical Cyclone Multi-Task Estimation](https://mlsanity.com/paper/2511.11750/) (2025)
- [Solving stiff dark matter equations via Jacobian Normalization with Physics-Informed Neural Networks](https://mlsanity.com/paper/2602.21988/) (2026)

## Abstract (arXiv)

Several recent works in scientific machine learning have revived interest in the application of neural networks to partial differential equations (PDEs). A popular approach is to aggregate the residual form of the governing PDE and its boundary conditions as soft penalties into a composite objective/loss function for training neural networks, which is commonly referred to as physics-informed neural networks (PINNs). In the present study, we visualize the loss landscapes and distributions of learned parameters and explain the ways this particular formulation of the objective function may hinder or even prevent convergence when dealing with challenging target solutions. We construct a purely data-driven loss function composed of both the boundary loss and the domain loss. Using this data-driven loss function and, separately, a physics-informed loss function, we then train two neural network models with the same architecture. We show that incomparable scales between boundary and domain loss terms are the culprit behind the poor performance. Additionally, we assess the performance of both approaches on two elliptic problems with increasingly complex target solutions. Based on our analysis of their loss landscapes and learned parameter distributions, we observe that a physics-informed neural network with a composite objective function formulation produces highly non-convex loss surfaces that are difficult to optimize and are more prone to the problem of vanishing gradients.