mlsanityData to 7 Oct 2026

Mitigating Propagation Failures in Physics-informed Neural Networks using Retain-Resample-Release (R3) Sampling

arXiv 2207.02338 ↗First posted 5 Jul 2022cs.LG cs.AIcited by 71

Not on the map. Why?

Abstract (arXiv)

Despite the success of physics-informed neural networks (PINNs) in approximating partial differential equations (PDEs), PINNs can sometimes fail to converge to the correct solution in problems involving complicated PDEs. This is reflected in several recent studies on characterizing the "failure modes" of PINNs, although a thorough understanding of the connection between PINN failure modes and sampling strategies is missing. In this paper, we provide a novel perspective of failure modes of PINNs by hypothesizing that training PINNs relies on successful "propagation" of solution from initial and/or boundary condition points to interior points. We show that PINNs with poor sampling strategies can get stuck at trivial solutions if there are propagation failures, characterized by highly imbalanced PDE residual fields. To mitigate propagation failures, we propose a novel Retain-Resample-Release sampling (R3) algorithm that can incrementally accumulate collocation points in regions of high PDE residuals with little to no computational overhead. We provide an extension of R3 sampling to respect the principle of causality while solving time-dependent PDEs. We theoretically analyze the behavior of R3 sampling and empirically demonstrate its efficacy and efficiency in comparison with baselines on a variety of PDE problems.

Key facts. Cited by 71 papers in the map (data through 7 Oct 2026). Builds on 14 of its 28 references that are in the map. First posted on arXiv 5 Jul 2022.

Builds on

14 of 28 references.

References · 14

  1. 2021Characterizing possible failure modes in physics-informed neural networkscited by 632
  2. 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 629
  3. 2019DeepXDE: A deep learning library for solving differential equationscited by 617
  4. 2017Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equationscited by 362
  5. 2022Respecting causality is all you need for training physics-informed neural networkscited by 154
  6. 2021Efficient training of physics-informed neural networks via importance samplingcited by 109
  7. 2021Learning in Sinusoidal Spaces with Physics-Informed Neural Networkscited by 61
  8. 2020Physics-informed deep learning for incompressible laminar flowscited by 54
  9. 2020Machine learning for metal additive manufacturing: Predicting temperature and melt pool fluid dynamics using physics-informed neural networkscited by 42
  10. 2022On the Role of Fixed Points of Dynamical Systems in Training Physics-Informed Neural Networkscited by 23
  11. 2022Is L2 Physics-Informed Loss Always Suitable for Training Physics-Informed Neural Network?cited by 23
  12. 2020Physics-informed neural networks for myocardial perfusion MRI quantificationcited by 18
2 more
  1. 2021PCE-PINNs: Physics-Informed Neural Networks for Uncertainty Propagation in Ocean Modelingcited by 8
  2. 2021Revisit Geophysical Imaging in A New View of Physics-informed Generative Adversarial Learningcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 22; 2025: 23; 2026: 22.

71 papers.

Sort:

By area:

  1. 2024From PINNs to PIKANs: Recent Advances in Physics-Informed Machine Learningcited by 73
  2. 2024ConFIG: Towards Conflict-free Training of Physics Informed Neural Networkscited by 43
  3. 2024RoPINN: Region Optimized Physics-Informed Neural Networkscited by 26
  4. 2023Stacked networks improve physics-informed training: applications to neural networks and deep operator networkscited by 21
  5. 2024Dual Cone Gradient Descent for Training Physics-Informed Neural Networkscited by 21
  6. 2024Separable DeepONet: Breaking the Curse of Dimensionality in Physics-Informed Machine Learningcited by 14
  7. 2024Knowledge-guided Machine Learning: Current Trends and Future Prospectscited by 13
  8. 2025FP64 is All You Need: Rethinking Failure Modes in Physics-Informed Neural Networkscited by 11
  9. 2025Evolutionary Optimization of Physics-Informed Neural Networks: Evo-PINN Frontiers and Opportunitiescited by 10
  10. 2023Better Neural PDE Solvers Through Data-Free Mesh Moverscited by 9
  11. 2024A physics-informed transformer neural operator for learning generalized solutions of initial boundary value problemscited by 6
  12. 2025ProPINN: Demystifying Propagation Failures in Physics-Informed Neural Networkscited by 6
38 more
  1. 2025Complex Physics-Informed Neural Networkcited by 6
  2. 2025A Variational Framework for Residual-Based Adaptivity in Neural PDE Solvers and Operator Learningcited by 6
  3. 2024RBF-PINN: Non-Fourier Positional Embedding in Physics-Informed Neural Networkscited by 5
  4. 2024Deep adaptive sampling for surrogate modeling without labeled datacited by 5
  5. 2024Physics-informed Neural Networks for Functional Differential Equations: Cylindrical Approximation and Its Convergence Guaranteescited by 5
  6. 2025Dual-Balancing for Physics-Informed Neural Networkscited by 5
  7. 2024PTPI-DL-ROMs: pre-trained physics-informed deep learning-based reduced order models for nonlinear parametrized PDEscited by 4
  8. 2024Beyond Derivative Pathology of PINNs: Variable Splitting Strategy with Convergence Analysiscited by 4
  9. 2025RAMS: Residual-based adversarial-gradient moving sample method for scientific machine learning in solving partial differential equationscited by 4
  10. 2024Element-wise Multiplication Based Deeper Physics-Informed Neural Networkscited by 3
  11. 2024Solving Differential Equations with Constrained Learningcited by 3
  12. 2025Enhanced Vascular Flow Simulations in Aortic Aneurysm via Physics-Informed Neural Networks and Deep Operator Networkscited by 3
  13. 2025PINNverse: Accurate parameter estimation in differential equations from noisy data with constrained physics-informed neural networkscited by 3
  14. 2025Integration Matters for Learning PDEs with Backward SDEscited by 3
  15. 2023Investigating the Ability of PINNs To Solve Burgers' PDE Near Finite-Time BlowUpcited by 2
  16. 2024Feature Mapping in Physics-Informed Neural Networks (PINNs)cited by 2
  17. 2024SetPINNs: Set-based Physics-informed Neural Networkscited by 2
  18. 2025Adaptive Collocation Point Strategies For Physics Informed Neural Networks via the QR Discrete Empirical Interpolation Methodcited by 2
  19. 2025Leveraging Influence Functions for Resampling Data in Physics-Informed Neural Networkscited by 2
  20. 2025AB-PINNs: Adaptive-Basis Physics-Informed Neural Networks for Residual-Driven Domain Decompositioncited by 2
  21. 2025PRISMA: Improving the Accuracy-Latency Frontier of Diffusion-based PDE Solvers Using Physics-Informed Spectral Attentioncited by 2
  22. 2024Capturing Shock Waves by Relaxation Neural Networkscited by 1
  23. 2024PINNfluence: Interpreting PINNs through Influence Functionscited by 1
  24. 2024MelissaDL x Breed: Towards Data-Efficient On-line Supervised Training of Multi-parametric Surrogates with Active Learningcited by 1
  25. 2025Neural Modular Physics for Elastic Simulationcited by 1
  26. 2026Cheap Thrills: Effective Amortized Optimization Using Inexpensive Labelscited by 1
  27. 2026Per-Loss Adapters for Gradient Conflict in Physics-Informed Neural Networkscited by 1
  28. 2026Error whitening: Why Gauss-Newton outperforms Newtoncited by 1
  29. 2026Neural Galerkin Normalizing Flows for Bayesian Inference of Diffusions with Inaccessible Boundariescited by 1
  30. 2026Decision-Aware Evaluation of Physics-Informed Surrogatescited by 1
  31. 2026Learning Where to Simulate: Generative Active Sampling for Online PDE Surrogate Trainingcited by 1
  32. 2023Dirichlet-Neumann learning algorithm for solving elliptic interface problemscited by 0
  33. 2024Learning Traveling Solitary Waves Using Separable Gaussian Neural Networkscited by 0
  34. 2024SGM-PINN: Sampling Graphical Models for Faster Training of Physics-Informed Neural Networkscited by 0
  35. 2024(Un)supervised Learning of Maximal Lyapunov Functionscited by 0
  36. 2025Provably Accurate Adaptive Sampling for Collocation Points in Physics-informed Neural Networkscited by 0
  37. 2025Reliable and efficient inverse analysis using physics-informed neural networks with normalized distance functions and adaptive weight tuningcited by 0
  38. 2025M'ethode de quadrature pour les PINNs fond'ee th'eoriquement sur la hessienne des r'esiduelscited by 0

50 most cited of 71 All 71 (JSON)