# Uncertainty propagation in auto-regressive random neural network models

arXiv: https://arxiv.org/abs/2608.20483 · HTML: https://mlsanity.com/paper/2608.20483/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 47 references that are in the map.
- First posted on arXiv 20 Aug 2026.

## Builds on (most cited first, 28 of 28)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges](https://mlsanity.com/paper/2011.06225/) (2020)
- [Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent](https://mlsanity.com/paper/1902.06720/) (2019)
- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [Weight Uncertainty in Neural Networks](https://mlsanity.com/paper/1505.05424/) (2015)
- [Time Series Forecasting With Deep Learning: A Survey](https://mlsanity.com/paper/2004.13408/) (2020)
- [Laplace Redux -- Effortless Bayesian Deep Learning](https://mlsanity.com/paper/2106.14806/) (2021)
- [Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users](https://mlsanity.com/paper/2007.06823/) (2020)
- [Explaining NonLinear Classification Decisions with Deep Taylor Decomposition](https://mlsanity.com/paper/1512.02479/) (2015)
- [Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1708.00588/) (2017)
- [Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons](https://mlsanity.com/paper/2201.07766/) (2022)
- [Bayesian Neural Networks: An Introduction and Survey](https://mlsanity.com/paper/2006.12024/) (2020)
- [PPINN: Parareal Physics-Informed Neural Network for time-dependent PDEs](https://mlsanity.com/paper/1909.10145/) (2019)
- [Uncertainty quantification for noisy inputs-outputs in physics-informed neural networks and neural operators](https://mlsanity.com/paper/2311.11262/) (2023)
- [A Primer on Bayesian Neural Networks: Review and Debates](https://mlsanity.com/paper/2309.16314/) (2023)
- [Uncertainty Quantification via Stable Distribution Propagation](https://mlsanity.com/paper/2402.08324/) (2024)
- [Neural Taylor Approximations: Convergence and Exploration in Rectifier Networks](https://mlsanity.com/paper/1611.02345/) (2016)
- [Uncertainty Propagation in Deep Neural Networks Using Extended Kalman Filtering](https://mlsanity.com/paper/1809.06009/) (2018)
- [Physics-informed Neural Networks for Functional Differential Equations: Cylindrical Approximation and Its Convergence Guarantees](https://mlsanity.com/paper/2410.18153/) (2024)
- [HopCast: Calibration of Autoregressive Dynamics Models](https://mlsanity.com/paper/2501.16587/) (2025)
- [Uncertainty propagation in feed-forward neural network models](https://mlsanity.com/paper/2503.21059/) (2025)
- [Polynomial Neural Networks and Taylor maps for Dynamical Systems Simulation and Learning](https://mlsanity.com/paper/1912.09986/) (2019)
- [HOPE: High-order Polynomial Expansion of Black-box Neural Networks](https://mlsanity.com/paper/2307.08192/) (2023)
- [Surrogate modeling for uncertainty quantification in nonlinear dynamics](https://mlsanity.com/paper/2507.12358/) (2025)
- [Assumed Density Filtering and Smoothing with Neural Network Surrogate Models](https://mlsanity.com/paper/2511.09016/) (2025)
- [Exact Gaussian Moment Matching for Residual Networks: a Second-Order Method](https://mlsanity.com/paper/2601.22307/) (2026)

## Abstract (arXiv)

We develop analytical and particle-based methods for uncertainty propagation in random neural network models, where both the inputs and network parameters are allowed to be random. Building on the piecewise-linear structure of the Leaky ReLU activation function, we derive a local approximation of the neural network output with respect to perturbations in both its inputs and parameters. This approximation is exact for perturbations that preserve the network activation pattern, and it allows us to compute analytical expressions for the probability density function and characteristic function of the network output, together with closed-form approximations for its mean and covariance. We extend this uncertainty propagation framework to autonomous dynamical systems whose one-step evolution map is represented by a random neural network. Repeated application of this map defines an autoregressive model, for which we derive recursive equations to propagate uncertainty in both the state and network parameters over time. These equations explicitly account for the state-parameter cross-covariance that develops under successive iterations of the network. Numerical experiments on the Lorenz-63 system and the Kuramoto-Sivashinsky equation demonstrate accurate uncertainty propagation through the predictability horizon and the applicability of the proposed framework to high-dimensional dynamical systems.