# Preconditioning for Physics-Informed Neural Networks

arXiv: https://arxiv.org/abs/2402.00531 · HTML: https://mlsanity.com/paper/2402.00531/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 40 references that are in the map.
- First posted on arXiv 1 Feb 2024.

## Builds on (most cited first, 25 of 25)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [When and why PINNs fail to train: A neural tangent kernel perspective](https://mlsanity.com/paper/2007.14527/) (2020)
- [DeepXDE: A deep learning library for solving differential equations](https://mlsanity.com/paper/1907.04502/) (2019)
- [Physics-informed neural networks (PINNs) for fluid mechanics: A review](https://mlsanity.com/paper/2105.09506/) (2021)
- [Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks](https://mlsanity.com/paper/1901.06523/) (2019)
- [B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data](https://mlsanity.com/paper/2003.06097/) (2020)
- [Physics-informed neural networks with hard constraints for inverse design](https://mlsanity.com/paper/2102.04626/) (2021)
- [Respecting causality is all you need for training physics-informed neural networks](https://mlsanity.com/paper/2203.07404/) (2022)
- [A unified deep artificial neural network approach to partial differential equations in complex geometries](https://mlsanity.com/paper/1711.06464/) (2017)
- [Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating a class of inverse problems for PDEs](https://mlsanity.com/paper/2007.01138/) (2020)
- [Physics-Informed Machine Learning: A Survey on Problems, Methods and Applications](https://mlsanity.com/paper/2211.08064/) (2022)
- [PINNacle: A Comprehensive Benchmark of Physics-Informed Neural Networks for Solving PDEs](https://mlsanity.com/paper/2306.08827/) (2023)
- [Physics-informed neural networks for inverse problems in supersonic flows](https://mlsanity.com/paper/2202.11821/) (2022)
- [Error estimates for physics informed neural networks approximating the Navier-Stokes equations](https://mlsanity.com/paper/2203.09346/) (2022)
- [Error analysis for physics informed neural networks (PINNs) approximating Kolmogorov PDEs](https://mlsanity.com/paper/2106.14473/) (2021)
- [Machine learning for metal additive manufacturing: Predicting temperature and melt pool fluid dynamics using physics-informed neural networks](https://mlsanity.com/paper/2008.13547/) (2020)
- [PFNN: A Penalty-Free Neural Network Method for Solving a Class of Second-Order Boundary-Value Problems on Complex Geometries](https://mlsanity.com/paper/2004.06490/) (2020)
- [Physics-informed Dyna-Style Model-Based Deep Reinforcement Learning for Dynamic Control](https://mlsanity.com/paper/2108.00128/) (2021)
- [Energy-based error bound of physics-informed neural network solutions in elasticity](https://mlsanity.com/paper/2010.09088/) (2020)
- [A Unified Hard-Constraint Framework for Solving Geometrically Complex PDEs](https://mlsanity.com/paper/2210.03526/) (2022)
- [Adversarial Ink: Componentwise Backward Error Attacks on Deep Learning](https://mlsanity.com/paper/2306.02918/) (2023)

## Built on by (most cited first, 13 of 13)

- [Guided Diffusion Sampling on Function Spaces with Applications to PDEs](https://mlsanity.com/paper/2505.17004/) (2025)
- [Kronecker-Factored Approximate Curvature for Physics-Informed Neural Networks](https://mlsanity.com/paper/2405.15603/) (2024)
- [ANaGRAM: A Natural Gradient Relative to Adapted Model for efficient PINNs learning](https://mlsanity.com/paper/2412.10782/) (2024)
- [Physics-Informed Neural Networks with Trust-Region Sequential Quadratic Programming](https://mlsanity.com/paper/2409.10777/) (2024)
- [PRISMA: Improving the Accuracy-Latency Frontier of Diffusion-based PDE Solvers Using Physics-Informed Spectral Attention](https://mlsanity.com/paper/2512.01370/) (2025)
- [A Natural Primal-Dual Hybrid Gradient Method for Adversarial Neural Network Training on Solving Partial Differential Equations](https://mlsanity.com/paper/2411.06278/) (2024)
- [Accelerating Natural Gradient Descent for PINNs with Randomized Numerical Linear Algebra](https://mlsanity.com/paper/2505.11638/) (2025)
- [FEM-Informed Hypergraph Neural Networks for Efficient Elastoplasticity](https://mlsanity.com/paper/2602.07364/) (2026)
- [Sparse Random-Feature Neural Networks with Krylov-Based SVD for Singularly Perturbed ODE](https://mlsanity.com/paper/2605.07286/) (2026)
- [PINNs Failure Modes are Overfitting](https://mlsanity.com/paper/2605.30910/) (2026)
- [SS-ESOAP: Self-Scaled Adaptive Preconditioning for Physics-Informed Learning](https://mlsanity.com/paper/2608.29448/) (2026)
- [GeoWind2Plan: Mission-Time 3D Urban Wind Prediction for Energy-Efficient UAV Planning](https://mlsanity.com/paper/2609.36056/) (2026)
- [An overview of machine learning-enhanced iterative methods for systems of linear and nonlinear equations](https://mlsanity.com/paper/2610.07211/) (2026)

## Abstract (arXiv)

Physics-informed neural networks (PINNs) have shown promise in solving various partial differential equations (PDEs). However, training pathologies have negatively affected the convergence and prediction accuracy of PINNs, which further limits their practical applications. In this paper, we propose to use condition number as a metric to diagnose and mitigate the pathologies in PINNs. Inspired by classical numerical analysis, where the condition number measures sensitivity and stability, we highlight its pivotal role in the training dynamics of PINNs. We prove theorems to reveal how condition number is related to both the error control and convergence of PINNs. Subsequently, we present an algorithm that leverages preconditioning to improve the condition number. Evaluations of 18 PDE problems showcase the superior performance of our method. Significantly, in 7 of these problems, our method reduces errors by an order of magnitude. These empirical findings verify the critical role of the condition number in PINNs' training.