# A Benchmark on Uncertainty Quantification for Deep Learning Prognostics

arXiv: https://arxiv.org/abs/2302.04730 · HTML: https://mlsanity.com/paper/2302.04730/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 95 references that are in the map.
- First posted on arXiv 9 Feb 2023.

## Builds on (most cited first, 30 of 31)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges](https://mlsanity.com/paper/2011.06225/) (2020)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [Variational Dropout and the Local Reparameterization Trick](https://mlsanity.com/paper/1506.02557/) (2015)
- [Bayesian Deep Learning and a Probabilistic Perspective of Generalization](https://mlsanity.com/paper/2002.08791/) (2020)
- [Accurate Uncertainties for Deep Learning Using Calibrated Regression](https://mlsanity.com/paper/1807.00263/) (2018)
- [Pyro: Deep Universal Probabilistic Programming](https://mlsanity.com/paper/1810.09538/) (2018)
- [Deep Evidential Regression](https://mlsanity.com/paper/1910.02600/) (2019)
- [Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks](https://mlsanity.com/paper/1502.05336/) (2015)
- [Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batches](https://mlsanity.com/paper/1803.04386/) (2018)
- [On the Pitfalls of Heteroscedastic Uncertainty Estimation with Probabilistic Neural Networks](https://mlsanity.com/paper/2203.09168/) (2022)
- [Potential, Challenges and Future Directions for Deep Learning in Prognostics and Health Management Applications](https://mlsanity.com/paper/2005.02144/) (2020)
- [Reliable training and estimation of variance networks](https://mlsanity.com/paper/1906.03260/) (2019)
- [Uncertainty Toolbox: an Open-Source Library for Assessing, Visualizing, and Improving Uncertainty Quantification](https://mlsanity.com/paper/2109.10254/) (2021)
- [Fusing Physics-based and Deep Learning Models for Prognostics](https://mlsanity.com/paper/2003.00732/) (2020)
- [Radial Bayesian Neural Networks: Beyond Discrete Support In Large-Scale Bayesian Deep Learning](https://mlsanity.com/paper/1907.00865/) (2019)
- [AutoDEUQ: Automated Deep Ensemble with Uncertainty Quantification](https://mlsanity.com/paper/2110.13511/) (2021)
- [Remaining Useful Lifetime Prediction via Deep Domain Adaptation](https://mlsanity.com/paper/1907.07480/) (2019)
- [Forecasting remaining useful life: Interpretable deep learning approach via variational Bayesian inferences](https://mlsanity.com/paper/1907.05146/) (2019)
- [Prior choice affects ability of Bayesian neural networks to identify unknowns](https://mlsanity.com/paper/2005.04987/) (2020)

## Built on by (most cited first, 9 of 9)

- [Asymmetric Duos: Sidekicks Improve Uncertainty](https://mlsanity.com/paper/2505.18636/) (2025)
- [Bayesian Physics Informed Neural Networks for Reliable Transformer Prognostics](https://mlsanity.com/paper/2509.15933/) (2025)
- [h-calibration: Rethinking Classifier Recalibration with Probabilistic Error-Bounded Objective](https://mlsanity.com/paper/2506.17968/) (2025)
- [Probabilistic Neural Networks (PNNs) with t-Distributed Outputs: Adaptive Prediction Intervals Beyond Gaussian Assumptions](https://mlsanity.com/paper/2503.12354/) (2025)
- [Decision-Aligned Evaluation of Uncertainty Quantification](https://mlsanity.com/paper/2606.26990/) (2026)
- [Prediction Accuracy & Reliability: Classification and Object Localization under Distribution Shift](https://mlsanity.com/paper/2409.03543/) (2024)
- [A Heterogeneous Long-Micro Scale Cascading Architecture for General Aviation Health Management](https://mlsanity.com/paper/2603.22885/) (2026)
- [WaveGraphNet: Physics-Consistent Guided-Wave Damage Localization through Coupled Inverse-Forward Graph Learning](https://mlsanity.com/paper/2605.20311/) (2026)
- [Bifurcated Remaining Useful Life Prediction: A Hybrid Approach for Realistic Uncertainty Characterization](https://mlsanity.com/paper/2605.31241/) (2026)

## Abstract (arXiv)

Reliable uncertainty quantification on RUL prediction is crucial for informative decision-making in predictive maintenance. In this context, we assess some of the latest developments in the field of uncertainty quantification for prognostics deep learning. This includes the state-of-the-art variational inference algorithms for Bayesian neural networks (BNN) as well as popular alternatives such as Monte Carlo Dropout (MCD), deep ensembles (DE) and heteroscedastic neural networks (HNN). All the inference techniques share the same inception deep learning architecture as a functional model. We performed hyperparameter search to optimize the main variational and learning parameters of the algorithms. The performance of the methods is evaluated on a subset of the large NASA NCMAPSS dataset for aircraft engines. The assessment includes RUL prediction accuracy, the quality of predictive uncertainty, and the possibility to break down the total predictive uncertainty into its aleatoric and epistemic parts. The results show no method clearly outperforms the others in all the situations. Although all methods are close in terms of accuracy, we find differences in the way they estimate uncertainty. Thus, DE and MCD generally provide more conservative predictive uncertainty than BNN. Surprisingly, HNN can achieve strong results without the added training complexity and extra parameters of the BNN. For tasks like active learning where a separation of epistemic and aleatoric uncertainty is required, radial BNN and MCD seem the best options.