# A Survey on Epistemic (Model) Uncertainty in Supervised Learning: Recent Advances and Applications

arXiv: https://arxiv.org/abs/2111.01968 · HTML: https://mlsanity.com/paper/2111.01968/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 68 of its 180 references that are in the map.
- First posted on arXiv 3 Nov 2021.

## Builds on (most cited first, 30 of 68)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Averaging Weights Leads to Wider Optima and Better Generalization](https://mlsanity.com/paper/1803.05407/) (2018)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [A Simple Baseline for Bayesian Uncertainty in Deep Learning](https://mlsanity.com/paper/1902.02476/) (2019)
- [Variational Dropout and the Local Reparameterization Trick](https://mlsanity.com/paper/1506.02557/) (2015)
- [Stochastic Gradient Hamiltonian Monte Carlo](https://mlsanity.com/paper/1402.4102/) (2014)
- [Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users](https://mlsanity.com/paper/2007.06823/) (2020)
- [Stochastic Gradient Descent as Approximate Bayesian Inference](https://mlsanity.com/paper/1704.04289/) (2017)
- [Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks](https://mlsanity.com/paper/1807.07356/) (2018)
- [Concrete Dropout](https://mlsanity.com/paper/1705.07832/) (2017)
- [Cyclical Stochastic Gradient MCMC for Bayesian Deep Learning](https://mlsanity.com/paper/1902.03932/) (2019)
- [Online Structured Laplace Approximations For Overcoming Catastrophic Forgetting](https://mlsanity.com/paper/1805.07810/) (2018)
- [A Review of Generalized Zero-Shot Learning Methods](https://mlsanity.com/paper/2011.08641/) (2020)
- [Practical Deep Learning with Bayesian Principles](https://mlsanity.com/paper/1906.02506/) (2019)
- [Exploring Uncertainty Measures in Deep Networks for Multiple Sclerosis Lesion Detection and Segmentation](https://mlsanity.com/paper/1808.01200/) (2018)
- [Bounding Box Regression with Uncertainty for Accurate Object Detection](https://mlsanity.com/paper/1809.08545/) (2018)
- [Evaluating Bayesian Deep Learning Methods for Semantic Segmentation](https://mlsanity.com/paper/1811.12709/) (2018)
- [Modelling Uncertainty in Deep Learning for Camera Relocalization](https://mlsanity.com/paper/1509.05909/) (2015)
- [Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adam](https://mlsanity.com/paper/1806.04854/) (2018)
- [A General Framework for Uncertainty Estimation in Deep Learning](https://mlsanity.com/paper/1907.06890/) (2019)
- [Uncertainty-aware Joint Salient Object and Camouflaged Object Detection](https://mlsanity.com/paper/2104.02628/) (2021)

## Built on by (most cited first, 24 of 24)

- [Trustworthy clinical AI solutions: a unified review of uncertainty quantification in deep learning models for medical image analysis](https://mlsanity.com/paper/2210.03736/) (2022)
- [A review of predictive uncertainty estimation with machine learning](https://mlsanity.com/paper/2209.08307/) (2022)
- [A Review of Deep Learning for Video Captioning](https://mlsanity.com/paper/2304.11431/) (2023)
- [Balancing policy constraint and ensemble size in uncertainty-based offline reinforcement learning](https://mlsanity.com/paper/2303.14716/) (2023)
- [Uncertainty Quantification via Stable Distribution Propagation](https://mlsanity.com/paper/2402.08324/) (2024)
- [DC-cycleGAN: Bidirectional CT-to-MR Synthesis from Unpaired Data](https://mlsanity.com/paper/2211.01293/) (2022)
- [ReliOcc: Towards Reliable Semantic Occupancy Prediction via Uncertainty Learning](https://mlsanity.com/paper/2409.18026/) (2024)
- [Functional-level Uncertainty Quantification for Calibrated Fine-tuning on LLMs](https://mlsanity.com/paper/2410.06431/) (2024)
- [Uncertainty Prioritized Experience Replay](https://mlsanity.com/paper/2506.09270/) (2025)
- [Geometric Transformation Uncertainty for Improving 3D Fetal Brain Pose Prediction from Freehand 2D Ultrasound Videos](https://mlsanity.com/paper/2405.13235/) (2024)
- [LTAU-FF: Loss Trajectory Analysis for Uncertainty in Atomistic Force Fields](https://mlsanity.com/paper/2402.00853/) (2024)
- [Do Compact SSL Backbones Matter for Audio Deepfake Detection? A Controlled Study with RAPTOR](https://mlsanity.com/paper/2603.06164/) (2026)
- [Deep Huber quantile regression networks](https://mlsanity.com/paper/2306.10306/) (2023)
- [Transferable Candidate Proposal with Bounded Uncertainty](https://mlsanity.com/paper/2312.04604/) (2023)
- [Uncertainty Aware Neural Network from Similarity and Sensitivity](https://mlsanity.com/paper/2304.14925/) (2023)
- [How False Data Affects Machine Learning Models in Electrochemistry?](https://mlsanity.com/paper/2311.10795/) (2023)
- [Leveraging Uncertainty Estimates To Improve Classifier Performance](https://mlsanity.com/paper/2311.11723/) (2023)
- [Last Layer Hamiltonian Monte Carlo](https://mlsanity.com/paper/2507.08905/) (2025)
- [Advancing Image Classification with Discrete Diffusion Classification Modeling](https://mlsanity.com/paper/2511.20263/) (2025)
- [SphUnc: Hyperspherical Uncertainty Decomposition and Causal Identification via Information Geometry](https://mlsanity.com/paper/2603.01168/) (2026)
- [Learning From High-Dimensional Cyber-Physical Data Streams for Diagnosing Faults in Smart Grids](https://mlsanity.com/paper/2303.08300/) (2023)
- [Model Uncertainty based Active Learning on Tabular Data using Boosted Trees](https://mlsanity.com/paper/2310.19573/) (2023)
- [Two-step dynamic obstacle avoidance](https://mlsanity.com/paper/2311.16841/) (2023)
- [Latent Uncertainty Representations for Video-based Driver Action and Intention Recognition](https://mlsanity.com/paper/2510.05006/) (2025)

## Abstract (arXiv)

Quantifying the uncertainty of supervised learning models plays an important role in making more reliable predictions. Epistemic uncertainty, which usually is due to insufficient knowledge about the model, can be reduced by collecting more data or refining the learning models. Over the last few years, scholars have proposed many epistemic uncertainty handling techniques which can be roughly grouped into two categories, i.e., Bayesian and ensemble. This paper provides a comprehensive review of epistemic uncertainty learning techniques in supervised learning over the last five years. As such, we, first, decompose the epistemic uncertainty into bias and variance terms. Then, a hierarchical categorization of epistemic uncertainty learning techniques along with their representative models is introduced. In addition, several applications such as computer vision (CV) and natural language processing (NLP) are presented, followed by a discussion on research gaps and possible future research directions.