# Understanding the Difficulty of Solving Cauchy Problems with PINNs

arXiv: https://arxiv.org/abs/2405.02561 · HTML: https://mlsanity.com/paper/2405.02561/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 38 references that are in the map.
- First posted on arXiv 4 May 2024.

## Builds on (most cited first, 15 of 15)

- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [When and why PINNs fail to train: A neural tangent kernel perspective](https://mlsanity.com/paper/2007.14527/) (2020)
- [Physics-informed neural networks (PINNs) for fluid mechanics: A review](https://mlsanity.com/paper/2105.09506/) (2021)
- [Neural Operator: Learning Maps Between Function Spaces](https://mlsanity.com/paper/2108.08481/) (2021)
- [Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism](https://mlsanity.com/paper/2009.04544/) (2020)
- [Deep Neural Network Approximation Theory](https://mlsanity.com/paper/1901.02220/) (2019)
- [Multifidelity deep neural operators for efficient learning of partial differential equations with application to fast inverse design of nanoscale heat transport](https://mlsanity.com/paper/2204.06684/) (2022)
- [Why Deep Neural Networks for Function Approximation?](https://mlsanity.com/paper/1610.04161/) (2016)
- [The Modern Mathematics of Deep Learning](https://mlsanity.com/paper/2105.04026/) (2021)
- [Neural Networks with Physics-Informed Architectures and Constraints for Dynamical Systems Modeling](https://mlsanity.com/paper/2109.06407/) (2021)
- [Bridging Physics-based and Data-driven modeling for Learning Dynamical Systems](https://mlsanity.com/paper/2011.10616/) (2020)
- [Experience report of physics-informed neural networks in fluid simulations: pitfalls and frustration](https://mlsanity.com/paper/2205.14249/) (2022)
- [Minimum width for universal approximation using ReLU networks on compact domain](https://mlsanity.com/paper/2309.10402/) (2023)
- [Blow up phenomena for gradient descent optimization methods in the training of artificial neural networks](https://mlsanity.com/paper/2211.15641/) (2022)

## Built on by (most cited first, 6 of 6)

- [Challenges and Advancements in Modeling Shock Fronts with Physics-Informed Neural Networks: A Review and Benchmarking Study](https://mlsanity.com/paper/2503.17379/) (2025)
- [Muon with Spectral Guidance: Efficient Optimization for Scientific Machine Learning](https://mlsanity.com/paper/2602.16167/) (2026)
- [Integration Matters for Learning PDEs with Backward SDEs](https://mlsanity.com/paper/2505.01078/) (2025)
- [AdamFLIP: Adaptive Momentum Feedback Linearization Optimization for Hard Constrained PINN Training](https://mlsanity.com/paper/2605.08408/) (2026)
- [Neural Quantum Spectral Operator Learning for Solving Partial Differential Equations](https://mlsanity.com/paper/2605.27408/) (2026)
- [A Graph-Based Inspection and Intervention Tool for Assessing Mechanistic Learning in PINNs](https://mlsanity.com/paper/2610.04939/) (2026)

## Abstract (arXiv)

Physics-Informed Neural Networks (PINNs) have gained popularity in scientific computing in recent years. However, they often fail to achieve the same level of accuracy as classical methods in solving differential equations. In this paper, we identify two sources of this issue in the case of Cauchy problems: the use of L2 residuals as objective functions and the approximation gap of neural networks. We show that minimizing the sum of L2 residual and initial condition error is not sufficient to guarantee the true solution, as this loss function does not capture the underlying dynamics. Additionally, neural networks are not capable of capturing singularities in the solutions due to the non-compactness of their image sets. This, in turn, influences the existence of global minima and the regularity of the network. We demonstrate that when the global minimum does not exist, machine precision becomes the predominant source of achievable error in practice. We also present numerical experiments in support of our theoretical claims.