mlsanityData to 14 Oct 2026

A Survey on Epistemic (Model) Uncertainty in Supervised Learning: Recent Advances and Applications

arXiv 2111.01968 ↗First posted 3 Nov 2021cs.LGcited by 24

Not on the map. Why?

Abstract (arXiv)

Quantifying the uncertainty of supervised learning models plays an important role in making more reliable predictions. Epistemic uncertainty, which usually is due to insufficient knowledge about the model, can be reduced by collecting more data or refining the learning models. Over the last few years, scholars have proposed many epistemic uncertainty handling techniques which can be roughly grouped into two categories, i.e., Bayesian and ensemble. This paper provides a comprehensive review of epistemic uncertainty learning techniques in supervised learning over the last five years. As such, we, first, decompose the epistemic uncertainty into bias and variance terms. Then, a hierarchical categorization of epistemic uncertainty learning techniques along with their representative models is introduced. In addition, several applications such as computer vision (CV) and natural language processing (NLP) are presented, followed by a discussion on research gaps and possible future research directions.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Builds on 68 of its 180 references that are in the map. First posted on arXiv 3 Nov 2021.

Builds on

68 of 180 references.

References · 68

  1. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  2. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  3. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  4. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
  5. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
  6. 2015Variational Inference with Normalizing Flowscited by 2,170
  7. 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,487
  8. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
  9. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,209
  10. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,013
  11. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 913
  12. 2021A Survey of Uncertainty in Deep Neural Networkscited by 680
38 more
  1. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 560
  2. 2015Variational Dropout and the Local Reparameterization Trickcited by 490
  3. 2014Stochastic Gradient Hamiltonian Monte Carlocited by 343
  4. 2020Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Userscited by 338
  5. 2017Stochastic Gradient Descent as Approximate Bayesian Inferencecited by 300
  6. 2018Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networkscited by 280
  7. 2017Concrete Dropoutcited by 227
  8. 2019Cyclical Stochastic Gradient MCMC for Bayesian Deep Learningcited by 203
  9. 2018Online Structured Laplace Approximations For Overcoming Catastrophic Forgettingcited by 201
  10. 2020A Review of Generalized Zero-Shot Learning Methodscited by 159
  11. 2019Practical Deep Learning with Bayesian Principlescited by 158
  12. 2018Exploring Uncertainty Measures in Deep Networks for Multiple Sclerosis Lesion Detection and Segmentationcited by 146
  13. 2018Bounding Box Regression with Uncertainty for Accurate Object Detectioncited by 143
  14. 2018Evaluating Bayesian Deep Learning Methods for Semantic Segmentationcited by 143
  15. 2015Modelling Uncertainty in Deep Learning for Camera Relocalizationcited by 134
  16. 2018Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adamcited by 131
  17. 2019A General Framework for Uncertainty Estimation in Deep Learningcited by 119
  18. 2021Uncertainty-aware Joint Salient Object and Camouflaged Object Detectioncited by 114
  19. 2017Noisy Natural Gradient as Variational Inferencecited by 94
  20. 2020Bayesian Deep Learning via Subnetwork Inferencecited by 87
  21. 2018Quantifying Uncertainties in Natural Language Processing Taskscited by 79
  22. 2018Bayesian Uncertainty Estimation for Batch Normalized Deep Networkscited by 70
  23. 2019Diversity with Cooperation: Ensemble Methods for Few-Shot Classificationcited by 70
  24. 2019Reducing Uncertainty in Undersampled MRI Reconstruction with Active Acquisitioncited by 56
  25. 2019A Bayesian Perspective on the Deep Image Priorcited by 52
  26. 2017Deep and Confident Prediction for Time Series at Ubercited by 50
  27. 2017Deep Forestcited by 46
  28. 2018Efficient Uncertainty Estimation for Semantic Segmentation in Videoscited by 46
  29. 2018Bayesian QuickNAT: Model Uncertainty in Deep Whole-Brain Segmentation for Structure-wise Quality Controlcited by 39
  30. 2019MonoLoco: Monocular 3D Pedestrian Localization and Uncertainty Estimationcited by 39
  31. 2019Uncertainty Estimation in One-Stage Object Detectioncited by 36
  32. 2018Structured Uncertainty Prediction Networkscited by 34
  33. 2018SLANG: Fast Structured Covariance Approximations for Bayesian Deep Learning with Natural Gradientcited by 34
  34. 2019Improving Back-Translation with Uncertainty-based Confidence Estimationcited by 34
  35. 2018Confidence Modeling for Neural Semantic Parsingcited by 33
  36. 2021Robust Representation Learning with Feedback for Single Image Derainingcited by 32
  37. 2020Hydra: Preserving Ensemble Diversity for Model Distillationcited by 31
  38. 2020Estimating Model Uncertainty of Neural Networks in Sparse Information Formcited by 31

50 most cited of 68 All 68 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 10; 2024: 5; 2025: 4; 2026: 2.

24 papers.

Sort:

By area:

  1. 2022Trustworthy clinical AI solutions: a unified review of uncertainty quantification in deep learning models for medical image analysiscited by 63
  2. 2022A review of predictive uncertainty estimation with machine learningcited by 29
  3. 2023A Review of Deep Learning for Video Captioningcited by 28
  4. 2023Balancing policy constraint and ensemble size in uncertainty-based offline reinforcement learningcited by 12
  5. 2024Uncertainty Quantification via Stable Distribution Propagationcited by 10
  6. 2022DC-cycleGAN: Bidirectional CT-to-MR Synthesis from Unpaired Datacited by 6
  7. 2024ReliOcc: Towards Reliable Semantic Occupancy Prediction via Uncertainty Learningcited by 6
  8. 2024Functional-level Uncertainty Quantification for Calibrated Fine-tuning on LLMscited by 6
  9. 2025Uncertainty Prioritized Experience Replaycited by 6
  10. 2024Geometric Transformation Uncertainty for Improving 3D Fetal Brain Pose Prediction from Freehand 2D Ultrasound Videoscited by 5
  11. 2024LTAU-FF: Loss Trajectory Analysis for Uncertainty in Atomistic Force Fieldscited by 4
  12. 2026Do Compact SSL Backbones Matter for Audio Deepfake Detection? A Controlled Study with RAPTORcited by 4
12 more
  1. 2023Deep Huber quantile regression networkscited by 3
  2. 2023Transferable Candidate Proposal with Bounded Uncertaintycited by 2
  3. 2023Uncertainty Aware Neural Network from Similarity and Sensitivitycited by 1
  4. 2023How False Data Affects Machine Learning Models in Electrochemistry?cited by 1
  5. 2023Leveraging Uncertainty Estimates To Improve Classifier Performancecited by 1
  6. 2025Last Layer Hamiltonian Monte Carlocited by 1
  7. 2025Advancing Image Classification with Discrete Diffusion Classification Modelingcited by 1
  8. 2026SphUnc: Hyperspherical Uncertainty Decomposition and Causal Identification via Information Geometrycited by 1
  9. 2023Learning From High-Dimensional Cyber-Physical Data Streams for Diagnosing Faults in Smart Gridscited by 0
  10. 2023Model Uncertainty based Active Learning on Tabular Data using Boosted Treescited by 0
  11. 2023Two-step dynamic obstacle avoidancecited by 0
  12. 2025Latent Uncertainty Representations for Video-based Driver Action and Intention Recognitioncited by 0