mlsanityData to 7 Oct 2026

The Epistemic Uncertainty Hole: an issue of Bayesian Neural Networks

arXiv 2407.01985 ↗First posted 2 Jul 2024stat.ML cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

Bayesian Deep Learning (BDL) gives access not only to aleatoric uncertainty, as standard neural networks already do, but also to epistemic uncertainty, a measure of confidence a model has in its own predictions. In this article, we show through experiments that the evolution of epistemic uncertainty metrics regarding the model size and the size of the training set, goes against theoretical expectations. More precisely, we observe that the epistemic uncertainty collapses literally in the presence of large models and sometimes also of little training data, while we expect the exact opposite behaviour. This phenomenon, which we call "epistemic uncertainty hole", is all the more problematic as it undermines the entire applicative potential of BDL, which is based precisely on the use of epistemic uncertainty. As an example, we evaluate the practical consequences of this uncertainty hole on one of the main applications of BDL, namely the detection of out-of-distribution samples

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 13 of its 19 references that are in the map. First posted on arXiv 2 Jul 2024.

Builds on

13 of 19 references.

References · 13

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2013Intriguing properties of neural networkscited by 5,114
  3. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  4. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  5. 2013Auto-Encoding Variational Bayescited by 3,980
  6. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  7. 2017Deep Bayesian Active Learning with Image Datacited by 812
  8. 2015Weight Uncertainty in Neural Networkscited by 554
  9. 2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
  10. 2021ResMLP: Feedforward networks for image classification with data-efficient trainingcited by 338
  11. 2018Understanding Measures of Uncertainty for Adversarial Example Detectioncited by 177
  12. 2019Estimating Risk and Uncertainty in Deep Reinforcement Learningcited by 68
1 more
  1. 2020A Comparison of Uncertainty Estimation Approaches in Deep Learning Components for Autonomous Vehicle Applicationscited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 5.

11 papers.

  1. 2024(Implicit) Ensembles of Ensembles: Epistemic Uncertainty Collapse in Large Modelscited by 15
  2. 2025Position: Epistemic uncertainty estimation methods are fundamentally incompletecited by 8
  3. 2025Reducing Aleatoric and Epistemic Uncertainty through Multi-modal Data Acquisitioncited by 5
  4. 2025From predictions to confidence intervals: an empirical study of conformal prediction methods for in-context learningcited by 2
  5. 2025Trust Me, I Know the Way: Predictive Uncertainty in the Presence of Shortcut Learningcited by 1
  6. 2025Scaling Laws for Uncertainty in Deep Learningcited by 1
  7. 2026Rethinking Uncertainty Quantification and Entanglement in Image Segmentationcited by 1
  8. 2026Do Deep Ensembles Actually Capture Uncertainty in Graph Neural Networks?cited by 0
  9. 2026Can we trust our models? Epistemic calibration in second-order classificationcited by 0
  10. 2026ST-LoRA: Single Trajectory LoRA Ensemble for Uncertainty Aware Agricultural Segmentationcited by 0
  11. 2026Structural Limits of the Information-Theoretic Uncertainty Decompositioncited by 0