# Are Statistical Methods Obsolete in the Era of Deep Learning? A Study of ODE Inverse Problems

arXiv: https://arxiv.org/abs/2505.21723 · HTML: https://mlsanity.com/paper/2505.21723/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 71 references that are in the map.
- First posted on arXiv 27 May 2025.

## Builds on (most cited first, 16 of 16)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [DeepXDE: A deep learning library for solving differential equations](https://mlsanity.com/paper/1907.04502/) (2019)
- [Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1711.10561/) (2017)
- [Machine Learning of Linear Differential Equations using Gaussian Processes](https://mlsanity.com/paper/1701.02440/) (2017)
- [Solving and Learning Nonlinear PDEs with Gaussian Processes](https://mlsanity.com/paper/2103.12959/) (2021)
- [An Expert's Guide to Training Physics-informed Neural Networks](https://mlsanity.com/paper/2308.08468/) (2023)
- [Challenges in Training PINNs: A Loss Landscape Perspective](https://mlsanity.com/paper/2402.01868/) (2024)
- [TensorFlow Distributions](https://mlsanity.com/paper/1711.10604/) (2017)
- [Approximate Bayesian Neural Operators: Uncertainty Quantification for Parametric PDEs](https://mlsanity.com/paper/2208.01565/) (2022)
- [Probabilistic Numerical Methods for PDE-constrained Bayesian Inverse Problems](https://mlsanity.com/paper/1701.04006/) (2017)
- [Inference of dynamic systems from noisy and sparse data via manifold-constrained Gaussian processes](https://mlsanity.com/paper/2009.07444/) (2020)
- [Physics-informed neural networks for myocardial perfusion MRI quantification](https://mlsanity.com/paper/2011.12844/) (2020)
- [APIK: Active Physics-Informed Kriging Model with Partial Differential Equations](https://mlsanity.com/paper/2012.11798/) (2020)
- [Bayesian Calibration of Imperfect Computer Models using Physics-Informed Priors](https://mlsanity.com/paper/2201.06463/) (2022)

## Abstract (arXiv)

In the era of AI, neural networks have become increasingly popular for modeling, inference, and prediction, largely due to their potential for universal approximation. With the proliferation of such deep learning models, a question arises: are leaner statistical methods still relevant? To shed insight on this question, we employ the mechanistic nonlinear ordinary differential equation (ODE) inverse problem as a testbed, using the physics-informed neural network (PINN) as a representative of the deep learning paradigm and manifold-constrained Gaussian process inference (MAGI) as a representative of statistically principled methods. Through case studies involving the SEIR model from epidemiology and the Lorenz model from chaotic dynamics, we demonstrate that statistical methods are far from obsolete, especially when working with sparse and noisy observations. On tasks such as parameter inference and trajectory reconstruction, statistically principled methods consistently achieve lower bias and variance, while using far fewer parameters and requiring less hyperparameter tuning. Statistical methods can also decisively outperform deep learning models on out-of-sample future prediction, where the absence of relevant data often leads overparameterized models astray. Additionally, we find that statistically principled approaches are more robust to accumulation of numerical imprecision and can represent the underlying system more faithfully to the true governing ODEs.