mlsanityData to 14 Oct 2026

Position: Epistemic uncertainty estimation methods are fundamentally incomplete

arXiv 2505.23506 ↗First posted 29 May 2025cs.LG stat.MLcited by 9

Not on the map. Why?

Abstract (arXiv)

Identifying and disentangling sources of predictive uncertainty is essential for trustworthy supervised learning. We argue that widely used second-order methods that disentangle aleatoric and epistemic uncertainty are fundamentally incomplete. First, we show that unaccounted bias contaminates uncertainty estimates by overestimating aleatoric (data-related) uncertainty and underestimating the epistemic (model-related) counterpart, leading to incorrect uncertainty quantification. Second, we demonstrate that existing methods capture only partial contributions to the variance-driven part of epistemic uncertainty; different approaches account for different variance sources, yielding estimates that are incomplete and difficult to interpret. Together, these results highlight that current epistemic uncertainty estimates can only be used in safety-critical and high-stakes decision-making when limitations are fully understood by end users and acknowledged by AI developers.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 51 of its 85 references that are in the map. First posted on arXiv 29 May 2025.

Builds on

51 of 85 references.

References · 51

  1. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  2. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  3. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  4. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
  5. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,485
  6. 2016Variational Inference: A Review for Statisticianscited by 1,472
  7. 2020Energy-based Out-of-distribution Detectioncited by 1,299
  8. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
  9. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 913
  10. 2017Selective Classification for Deep Neural Networkscited by 852
  11. 2017Deep Bayesian Active Learning with Image Datacited by 819
  12. 2018Evidential Deep Learning to Quantify Classification Uncertaintycited by 732
38 more
  1. 2021A Survey of Uncertainty in Deep Neural Networkscited by 680
  2. 2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 600
  3. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 560
  4. 2018Predictive Uncertainty Estimation via Prior Networkscited by 557
  5. 2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
  6. 2020Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awarenesscited by 389
  7. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 365
  8. 2019Deep Evidential Regressioncited by 355
  9. 2018Understanding Measures of Uncertainty for Adversarial Example Detectioncited by 177
  10. 2022Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisonscited by 170
  11. 2020Posterior Network: Uncertainty Estimation without OOD Samples via Density-Based Pseudo-Countscited by 164
  12. 2022Quantifying Aleatoric and Epistemic Uncertainty in Machine Learning: Are Conditional Entropy and Mutual Information Appropriate Measures?cited by 125
  13. 2022On the Pitfalls of Heteroscedastic Uncertainty Estimation with Probabilistic Neural Networkscited by 120
  14. 2019Reverse KL-Divergence Training of Prior Networks: Improved Uncertainty and Adversarial Robustnesscited by 104
  15. 2021Priors in Bayesian Deep Learning: A Reviewcited by 101
  16. 2022A Deeper Look into Aleatoric and Epistemic Uncertainty Disentanglementcited by 87
  17. 2023A Survey on Uncertainty Quantification Methods for Deep Learningcited by 81
  18. 2021Graph Posterior Network: Bayesian Predictive Uncertainty for Node Classificationcited by 80
  19. 2024Benchmarking Uncertainty Disentanglement: Specialized Uncertainties for Specialized Taskscited by 78
  20. 2021Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimationcited by 71
  21. 2021On the Practicality of Deterministic Epistemic Uncertaintycited by 62
  22. 2023Uncertainty Estimation by Fisher Information-based Evidential Deep Learningcited by 58
  23. 2023Credal Bayesian Deep Learningcited by 45
  24. 2024Rethinking Aleatoric and Epistemic Uncertaintycited by 42
  25. 2022The Unreasonable Effectiveness of Deep Evidential Regressioncited by 40
  26. 2020Evaluating Robustness of Predictive Uncertainty Estimation: Are Dirichlet-based Models Reliable?cited by 39
  27. 2023Second-Order Uncertainty Quantification: A Distance-Based Approachcited by 35
  28. 2024Is Epistemic Uncertainty Faithfully Represented by Evidential Deep Learning Methods?cited by 35
  29. 2025Position: Uncertainty Quantification Needs Reassessment for Large-language Model Agentscited by 32
  30. 2024Credal Learning Theorycited by 31
  31. 2024Quantifying Aleatoric and Epistemic Uncertainty with Proper Scoring Rulescited by 21
  32. 2022Uncertainty Estimates of Predictions via a General Bias-Variance Decompositioncited by 20
  33. 2023Integrating Uncertainty Awareness into Conformalized Quantile Regressioncited by 17
  34. 2021Natural Posterior Network: Deep Bayesian Uncertainty for Exponential Family Distributionscited by 13
  35. 2022Understanding the bias-variance tradeoff of Bregman divergencescited by 11
  36. 2024The Epistemic Uncertainty Hole: an issue of Bayesian Neural Networkscited by 11
  37. 2023Efficient Uncertainty Quantification and Reduction for Over-Parameterized Neural Networkscited by 10
  38. 2023One step closer to unbiased aleatoric uncertainty estimationcited by 10

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 8.

9 papers.

  1. 2026Uncertainty in Physics and AI: Taxonomy, Quantification, and Validationcited by 3
  2. 2026Structure-Aware Epistemic Uncertainty Quantification for Neural Operator PDE Surrogatescited by 2
  3. 2026Evaluating Epistemic Uncertainty: Beyond OOD Detection and Active Learningcited by 2
  4. 2025Multidimensional Uncertainty Quantification via Optimal Transportcited by 1
  5. 2026Subjective Risk Decomposition: A New View for Uncertainty Quantificationcited by 1
  6. 2026A Ground-Truth Framework for Uncertainty Disentanglement with Posterior Riskcited by 1
  7. 2026Can we trust our models? Epistemic calibration in second-order classificationcited by 0
  8. 2026Optimal Transport Dropout for Structured Predictive Uncertaintycited by 0
  9. 2026Rethinking Epistemic Uncertainty in Node Classification through Information Growthcited by 0