# Recipes for when Physics Fails: Recovering Robust Learning of Physics Informed Neural Networks

arXiv: https://arxiv.org/abs/2110.13330 · HTML: https://mlsanity.com/paper/2110.13330/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 48 references that are in the map.
- First posted on arXiv 26 Oct 2021.

## Builds on (most cited first, 15 of 15)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data](https://mlsanity.com/paper/2003.06097/) (2020)
- [Variational Physics-Informed Neural Networks For Solving Partial Differential Equations](https://mlsanity.com/paper/1912.00873/) (2019)
- [Solving and Learning Nonlinear PDEs with Gaussian Processes](https://mlsanity.com/paper/2103.12959/) (2021)
- [Adversarial Uncertainty Quantification in Physics-Informed Neural Networks](https://mlsanity.com/paper/1811.04026/) (2018)
- [PPINN: Parareal Physics-Informed Neural Network for time-dependent PDEs](https://mlsanity.com/paper/1909.10145/) (2019)
- [Machine learning for molecular simulation](https://mlsanity.com/paper/1911.02792/) (2019)
- [Structure Discovery in Nonparametric Regression through Compositional Kernel Search](https://mlsanity.com/paper/1302.4922/) (2013)
- [Numerical Gaussian Processes for Time-dependent and Non-linear Partial Differential Equations](https://mlsanity.com/paper/1703.10230/) (2017)
- [The Variational Gaussian Process](https://mlsanity.com/paper/1511.06499/) (2015)
- [Robustness Guarantees for Bayesian Inference with Gaussian Processes](https://mlsanity.com/paper/1809.06452/) (2018)
- [Adversarial Robustness Guarantees for Classification with Gaussian Processes](https://mlsanity.com/paper/1905.11876/) (2019)
- [Adaptive Inducing Points Selection For Gaussian Processes](https://mlsanity.com/paper/2107.10066/) (2021)
- [Sample-efficient reinforcement learning using deep Gaussian processes](https://mlsanity.com/paper/2011.01226/) (2020)

## Built on by (most cited first, 18 of 18)

- [ConFIG: Towards Conflict-free Training of Physics Informed Neural Networks](https://mlsanity.com/paper/2408.11104/) (2024)
- [Stacked networks improve physics-informed training: applications to neural networks and deep operator networks](https://mlsanity.com/paper/2311.06483/) (2023)
- [A multifidelity approach to continual learning for physical systems](https://mlsanity.com/paper/2304.03894/) (2023)
- [Flow Factorized Representation Learning](https://mlsanity.com/paper/2309.13167/) (2023)
- [Spectrally Adapted Physics-Informed Neural Networks for Solving Unbounded Domain Problems](https://mlsanity.com/paper/2202.02710/) (2022)
- [Physics-informed Neural Networks with Unknown Measurement Noise](https://mlsanity.com/paper/2211.15498/) (2022)
- [Adaptive Uncertainty-Guided Model Selection for Data-Driven PDE Discovery](https://mlsanity.com/paper/2308.10283/) (2023)
- [Stable Weight Updating: A Key to Reliable PDE Solutions Using Deep Learning](https://mlsanity.com/paper/2407.07375/) (2024)
- [Automatic Grid Updates for Kolmogorov-Arnold Networks using Layer Histograms](https://mlsanity.com/paper/2511.08570/) (2025)
- [Unsupervised Representation Learning from Sparse Transformation Analysis](https://mlsanity.com/paper/2410.05564/) (2024)
- [PHAST: Port-Hamiltonian Architecture for Structured Temporal Dynamics Forecasting](https://mlsanity.com/paper/2602.17998/) (2026)
- [Silent Failures in Physics-Informed Neural Networks: Parameter Poisoning and the Limits of Loss-Based Validation](https://mlsanity.com/paper/2606.25151/) (2026)
- [Dirichlet-Neumann learning algorithm for solving elliptic interface problems](https://mlsanity.com/paper/2301.07361/) (2023)
- [PINNing Cerebral Blood Flow: Analysis of Perfusion MRI in Infants using Physics-Informed Neural Networks](https://mlsanity.com/paper/2410.19759/) (2024)
- [Provably Accurate Adaptive Sampling for Collocation Points in Physics-informed Neural Networks](https://mlsanity.com/paper/2504.00910/) (2025)
- [M'ethode de quadrature pour les PINNs fond'ee th'eoriquement sur la hessienne des r'esiduels](https://mlsanity.com/paper/2506.20441/) (2025)
- [Overcoming the Limits of Finite Difference Method; Physics-Informed Neural Network for Noisy High-Dimensional Heat Diffusion](https://mlsanity.com/paper/2606.07982/) (2026)
- [Gradient Surgery for Physics-Informed Neural Networks](https://mlsanity.com/paper/2609.30966/) (2026)

## Abstract (arXiv)

Physics-informed Neural Networks (PINNs) have been shown to be effective in solving partial differential equations by capturing the physics induced constraints as a part of the training loss function. This paper shows that a PINN can be sensitive to errors in training data and overfit itself in dynamically propagating these errors over the domain of the solution of the PDE. It also shows how physical regularizations based on continuity criteria and conservation laws fail to address this issue and rather introduce problems of their own causing the deep network to converge to a physics-obeying local minimum instead of the global minimum. We introduce Gaussian Process (GP) based smoothing that recovers the performance of a PINN and promises a robust architecture against noise/errors in measurements. Additionally, we illustrate an inexpensive method of quantifying the evolution of uncertainty based on the variance estimation of GPs on boundary data. Robust PINN performance is also shown to be achievable by choice of sparse sets of inducing points based on sparsely induced GPs. We demonstrate the performance of our proposed methods and compare the results from existing benchmark models in literature for time-dependent Schr"odinger and Burgers' equations.