mlsanityData to 7 Oct 2026

FP64 is All You Need: Rethinking Failure Modes in Physics-Informed Neural Networks

arXiv 2505.10949 ↗First posted 16 May 2025cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

Physics Informed Neural Networks (PINNs) often exhibit failure modes in which the PDE residual loss converges while the solution error stays large, a phenomenon traditionally blamed on local optima separated from the true solution by steep loss barriers. We challenge this understanding by demonstrate that the real culprit is insufficient arithmetic precision: with standard FP32, the LBFGS optimizer prematurely satisfies its convergence test, freezing the network in a spurious failure phase. Simply upgrading to FP64 rescues optimization, enabling vanilla PINNs to solve PDEs without any failure modes. These results reframe PINN failure modes as precision induced stalls rather than inescapable local minima and expose a three stage training dynamic unconverged, failure, success whose boundaries shift with numerical precision. Our findings emphasize that rigorous arithmetic precision is the key to dependable PDE solving with neural networks.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 24 of its 38 references that are in the map. First posted on arXiv 16 May 2025.

Builds on

24 of 38 references.

References · 24

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2015Automatic differentiation in machine learning: a surveycited by 846
  3. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  4. 2024KAN: Kolmogorov-Arnold Networkscited by 634
  5. 2021Characterizing possible failure modes in physics-informed neural networkscited by 632
  6. 2019Linear Mode Connectivity and the Lottery Ticket Hypothesiscited by 627
  7. 2022Git Re-Basin: Merging Models modulo Permutation Symmetriescited by 460
  8. 2015Deep Learning with Limited Numerical Precisioncited by 368
  9. 2022A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networkscited by 233
  10. 2024Challenges in Training PINNs: A Loss Landscape Perspectivecited by 109
  11. 2022Mitigating Propagation Failures in Physics-informed Neural Networks using Retain-Resample-Release (R3) Samplingcited by 71
  12. 2024PirateNets: Physics-informed Deep Learning with Residual Adaptive Networkscited by 70
12 more
  1. 2022Failure-informed adaptive sampling for PINNscited by 64
  2. 2024Scaling FP8 training to trillion-token LLMscited by 63
  3. 2023PINNsFormer: A Transformer-Based Framework For Physics-Informed Neural Networkscited by 62
  4. 2021Quadratic Residual Networks: A New Class of Neural Networks for Solving Forward and Inverse Problems in Physics Involving PDEscited by 42
  5. 2024Parameterized Physics-informed Neural Networks for Parameterized PDEscited by 35
  6. 2024RoPINN: Region Optimized Physics-Informed Neural Networkscited by 26
  7. 2022Is L2 Physics-Informed Loss Always Suitable for Training Physics-Informed Neural Network?cited by 23
  8. 2022Bi-level Physics-Informed Neural Networks for PDE Constrained Optimization using Broyden's Hypergradientscited by 17
  9. 2022Critical Investigation of Failure Modes in Physics-informed Neural Networkscited by 15
  10. 2024PARCv2: Physics-aware Recurrent Convolutional Neural Networks for Spatiotemporal Dynamics Modelingcited by 10
  11. 2025Sub-Sequential Physics-Informed Learning with State Space Modelcited by 10
  12. 2025ProPINN: Demystifying Propagation Failures in Physics-Informed Neural Networkscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 9.

11 papers.

  1. 2026Curvature-Aware Optimization for High-Accuracy Physics-Informed Neural Networkscited by 11
  2. 2025A Variational Framework for Residual-Based Adaptivity in Neural PDE Solvers and Operator Learningcited by 6
  3. 2026Physics-Informed Modeling for Wood Thermal Analysis and Predictioncited by 1
  4. 2025Estimating Parameter Fields in Multi-Physics PDEs from Scarce Measurementscited by 0
  5. 2026PINNs Failure Modes are Overfittingcited by 0
  6. 2026Physics-Informed Neural Network with Squeeze-Excitation-like Attentioncited by 0
  7. 2026Tensor Train Diffusion: Leveraging Low-Rank Structures for High-Dimensional Score-Based Samplingcited by 0
  8. 2026Neither Precision Nor Architecture Alone: Controlled Tests of Failure Remedies for Physics-Informed Neural Networkscited by 0
  9. 2026SS-ESOAP: Self-Scaled Adaptive Preconditioning for Physics-Informed Learningcited by 0
  10. 2026SoftServe: A Scalable Quasi-Newton Method for Deep Learningcited by 0
  11. 2026Transferability of Learned States in Neural PDE Solverscited by 0