mlsanityData to 7 Oct 2026

Preconditioning for Physics-Informed Neural Networks

arXiv 2402.00531 ↗First posted 1 Feb 2024cs.LG cs.NA math.NAcited by 12

Not on the map. Why?

Abstract (arXiv)

Physics-informed neural networks (PINNs) have shown promise in solving various partial differential equations (PDEs). However, training pathologies have negatively affected the convergence and prediction accuracy of PINNs, which further limits their practical applications. In this paper, we propose to use condition number as a metric to diagnose and mitigate the pathologies in PINNs. Inspired by classical numerical analysis, where the condition number measures sensitivity and stability, we highlight its pivotal role in the training dynamics of PINNs. We prove theorems to reveal how condition number is related to both the error control and convergence of PINNs. Subsequently, we present an algorithm that leverages preconditioning to improve the condition number. Evaluations of 18 PDE problems showcase the superior performance of our method. Significantly, in 7 of these problems, our method reduces errors by an order of magnitude. These empirical findings verify the critical role of the condition number in PINNs' training.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 25 of its 40 references that are in the map. First posted on arXiv 1 Feb 2024.

Builds on

25 of 40 references.

References · 25

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  3. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  4. 2018On the Spectral Bias of Neural Networkscited by 1,321
  5. 2021Characterizing possible failure modes in physics-informed neural networkscited by 632
  6. 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 629
  7. 2019DeepXDE: A deep learning library for solving differential equationscited by 617
  8. 2021Physics-informed neural networks (PINNs) for fluid mechanics: A reviewcited by 470
  9. 2019Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networkscited by 360
  10. 2020B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Datacited by 329
  11. 2021Physics-informed neural networks with hard constraints for inverse designcited by 277
  12. 2022Respecting causality is all you need for training physics-informed neural networkscited by 154
13 more
  1. 2017A unified deep artificial neural network approach to partial differential equations in complex geometriescited by 122
  2. 2020Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating a class of inverse problems for PDEscited by 101
  3. 2022Physics-Informed Machine Learning: A Survey on Problems, Methods and Applicationscited by 88
  4. 2023PINNacle: A Comprehensive Benchmark of Physics-Informed Neural Networks for Solving PDEscited by 84
  5. 2022Physics-informed neural networks for inverse problems in supersonic flowscited by 75
  6. 2022Error estimates for physics informed neural networks approximating the Navier-Stokes equationscited by 63
  7. 2021Error analysis for physics informed neural networks (PINNs) approximating Kolmogorov PDEscited by 56
  8. 2020Machine learning for metal additive manufacturing: Predicting temperature and melt pool fluid dynamics using physics-informed neural networkscited by 42
  9. 2020PFNN: A Penalty-Free Neural Network Method for Solving a Class of Second-Order Boundary-Value Problems on Complex Geometriescited by 38
  10. 2021Physics-informed Dyna-Style Model-Based Deep Reinforcement Learning for Dynamic Controlcited by 22
  11. 2020Energy-based error bound of physics-informed neural network solutions in elasticitycited by 15
  12. 2022A Unified Hard-Constraint Framework for Solving Geometrically Complex PDEscited by 11
  13. 2023Adversarial Ink: Componentwise Backward Error Attacks on Deep Learningcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 3; 2026: 5.

12 papers.

  1. 2025Guided Diffusion Sampling on Function Spaces with Applications to PDEscited by 33
  2. 2024Kronecker-Factored Approximate Curvature for Physics-Informed Neural Networkscited by 21
  3. 2024ANaGRAM: A Natural Gradient Relative to Adapted Model for efficient PINNs learningcited by 7
  4. 2024Physics-Informed Neural Networks with Trust-Region Sequential Quadratic Programmingcited by 4
  5. 2024A Natural Primal-Dual Hybrid Gradient Method for Adversarial Neural Network Training on Solving Partial Differential Equationscited by 2
  6. 2025PRISMA: Improving the Accuracy-Latency Frontier of Diffusion-based PDE Solvers Using Physics-Informed Spectral Attentioncited by 2
  7. 2025Accelerating Natural Gradient Descent for PINNs with Randomized Numerical Linear Algebracited by 1
  8. 2026FEM-Informed Hypergraph Neural Networks for Efficient Elastoplasticitycited by 1
  9. 2026Sparse Random-Feature Neural Networks with Krylov-Based SVD for Singularly Perturbed ODEcited by 0
  10. 2026PINNs Failure Modes are Overfittingcited by 0
  11. 2026SS-ESOAP: Self-Scaled Adaptive Preconditioning for Physics-Informed Learningcited by 0
  12. 2026GeoWind2Plan: Mission-Time 3D Urban Wind Prediction for Energy-Efficient UAV Planningcited by 0