# Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorial

arXiv: https://arxiv.org/abs/2305.04933 · HTML: https://mlsanity.com/paper/2305.04933/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 81 of its 321 references that are in the map.
- First posted on arXiv 7 May 2023.

## Builds on (most cited first, 30 of 81)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges](https://mlsanity.com/paper/2011.06225/) (2020)
- [Evidential Deep Learning to Quantify Classification Uncertainty](https://mlsanity.com/paper/1806.01768/) (2018)
- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next](https://mlsanity.com/paper/2201.05624/) (2022)
- [Predictive Uncertainty Estimation via Prior Networks](https://mlsanity.com/paper/1802.10501/) (2018)
- [Physics-informed neural networks (PINNs) for fluid mechanics: A review](https://mlsanity.com/paper/2105.09506/) (2021)
- [Deep Kernel Learning](https://mlsanity.com/paper/1511.02222/) (2015)
- [Accurate Uncertainties for Deep Learning Using Calibrated Regression](https://mlsanity.com/paper/1807.00263/) (2018)
- [Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness](https://mlsanity.com/paper/2006.10108/) (2020)
- [Transfusion: Understanding Transfer Learning for Medical Imaging](https://mlsanity.com/paper/1902.07208/) (2019)
- [Laplace Redux -- Effortless Bayesian Deep Learning](https://mlsanity.com/paper/2106.14806/) (2021)
- [Deep Evidential Regression](https://mlsanity.com/paper/1910.02600/) (2019)
- [Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users](https://mlsanity.com/paper/2007.06823/) (2020)
- [Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data](https://mlsanity.com/paper/1901.06314/) (2019)
- [Advances in Variational Inference](https://mlsanity.com/paper/1711.05597/) (2017)
- [Concrete Dropout](https://mlsanity.com/paper/1705.07832/) (2017)

## Built on by (most cited first, 27 of 27)

- [Uncertainty Quantification on Clinical Trial Outcome Prediction](https://mlsanity.com/paper/2401.03482/) (2024)
- [Uncertainty Quantification in Large Language Models Through Convex Hull Analysis](https://mlsanity.com/paper/2406.19712/) (2024)
- [Uncertainty Quantification with the Empirical Neural Tangent Kernel](https://mlsanity.com/paper/2502.02870/) (2025)
- [Graph Evidential Learning for Anomaly Detection](https://mlsanity.com/paper/2506.00594/) (2025)
- [DoFlow: Flow-based Generative Models for Interventional and Counterfactual Forecasting on Time Series](https://mlsanity.com/paper/2511.02137/) (2025)
- [Improving Medical Diagnostics with Vision-Language Models: Convex Hull-Based Uncertainty Analysis](https://mlsanity.com/paper/2412.00056/) (2024)
- [Bayesian Physics Informed Neural Networks for Reliable Transformer Prognostics](https://mlsanity.com/paper/2509.15933/) (2025)
- [Probabilistic Neural Networks (PNNs) for Modeling Aleatoric Uncertainty in Scientific Machine Learning](https://mlsanity.com/paper/2402.13945/) (2024)
- [Old Rules in a New Game: Mapping Uncertainty Quantification to Quantum Machine Learning](https://mlsanity.com/paper/2507.14919/) (2025)
- [Is the Last Layer Sufficient for Uncertainty Quantification?](https://mlsanity.com/paper/2605.30741/) (2026)
- [Perturbation-Assisted Sample Synthesis: A Novel Approach for Uncertainty Quantification](https://mlsanity.com/paper/2305.18671/) (2023)
- [A comparison of machine learning surrogate models of street-scale flooding in Norfolk, Virginia](https://mlsanity.com/paper/2307.14185/) (2023)
- [Towards a Probabilistic Fusion Approach for Robust Battery Prognostics](https://mlsanity.com/paper/2405.15292/) (2024)
- [Legitimate ground-truth-free metrics for deep uncertainty classification scoring](https://mlsanity.com/paper/2410.23046/) (2024)
- [Uncertainty in Supply Chain Digital Twins: A Quantum-Classical Hybrid Approach](https://mlsanity.com/paper/2411.10254/) (2024)
- [Diffusion Model-based Parameter Estimation in Dynamic Power Systems](https://mlsanity.com/paper/2411.10431/) (2024)
- [Expert-elicitation method for non-parametric joint priors using normalizing flows](https://mlsanity.com/paper/2411.15826/) (2024)
- [A Priori Uncertainty Quantification of Reacting Turbulence Closure Models using Bayesian Neural Networks](https://mlsanity.com/paper/2402.18729/) (2024)
- [Towards Out-of-Distribution Detection for breast cancer classification in Point-of-Care Ultrasound Imaging](https://mlsanity.com/paper/2402.18960/) (2024)
- [Navigating Uncertainties in Machine Learning for Structural Dynamics: A Comprehensive Survey of Probabilistic and Non-Probabilistic Approaches in Forward and Inverse Problems](https://mlsanity.com/paper/2408.08629/) (2024)
- [Accounting for Uncertainty in Machine Learning Surrogates: A Gauss-Hermite Quadrature Approach to Reliability Analysis](https://mlsanity.com/paper/2509.18128/) (2025)
- [Uncertainty Guided Online Ensemble for Non-stationary Data Streams in Fusion Science](https://mlsanity.com/paper/2511.02092/) (2025)
- [Data-Driven Priors for Uncertainty-Aware Risk Prediction of Clinical Deterioration using Multimodal Data](https://mlsanity.com/paper/2603.08459/) (2026)
- [A Hybrid Intelligent Framework for Uncertainty-Aware Condition Monitoring of Industrial Systems](https://mlsanity.com/paper/2604.09932/) (2026)
- [Bifurcated Remaining Useful Life Prediction: A Hybrid Approach for Realistic Uncertainty Characterization](https://mlsanity.com/paper/2605.31241/) (2026)
- [In-Context Residual Calibration for Uncertainty Quantification of Energy Time Series over Graphs](https://mlsanity.com/paper/2606.31804/) (2026)
- [Uncertainty quantification for trustworthy deep learning: Methods and measures](https://mlsanity.com/paper/2607.28248/) (2026)

## Abstract (arXiv)

On top of machine learning models, uncertainty quantification (UQ) functions as an essential layer of safety assurance that could lead to more principled decision making by enabling sound risk assessment and management. The safety and reliability improvement of ML models empowered by UQ has the potential to significantly facilitate the broad adoption of ML solutions in high-stakes decision settings, such as healthcare, manufacturing, and aviation, to name a few. In this tutorial, we aim to provide a holistic lens on emerging UQ methods for ML models with a particular focus on neural networks and the applications of these UQ methods in tackling engineering design as well as prognostics and health management problems. Toward this goal, we start with a comprehensive classification of uncertainty types, sources, and causes pertaining to UQ of ML models. Next, we provide a tutorial-style description of several state-of-the-art UQ methods: Gaussian process regression, Bayesian neural network, neural network ensemble, and deterministic UQ methods focusing on spectral-normalized neural Gaussian process. Established upon the mathematical formulations, we subsequently examine the soundness of these UQ methods quantitatively and qualitatively (by a toy regression example) to examine their strengths and shortcomings from different dimensions. Then, we review quantitative metrics commonly used to assess the quality of predictive uncertainty in classification and regression problems. Afterward, we discuss the increasingly important role of UQ of ML models in solving challenging problems in engineering design and health prognostics. Two case studies with source codes available on GitHub are used to demonstrate these UQ methods and compare their performance in the life prediction of lithium-ion batteries at the early stage and the remaining useful life prediction of turbofan engines.