mlsanityData to 14 Oct 2026

Epistemic Uncertainty Is Not the Reducible Kind

arXiv 2606.12646 ↗First posted 10 Jun 2026stat.ML cs.IT cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

The standard taxonomy of predictive uncertainty defines epistemic uncertainty as the part removable by collecting more data, while the standard measure identifies it with a mutual-information term. We prove the definition and the measure are extensionally inconsistent. On an explicit construction, the measure assigns all uncertainty to the epistemic class, yet no quantity of training data reduces it. Reducibility is instead a property of the pair (uncertainty, acquisition class), and the dichotomy resolves into three parts: aleatoric, sample-reducible epistemic, and mechanism-reducible epistemic uncertainty. An exact identity for the value of an observation shows that in-distribution data never reduces mechanism-irreducible uncertainty and generically increases it. Ensemble disagreement, the deployed epistemic estimate, tracks the training procedure rather than the epistemic term. It collapses to zero beneath a positive truth under consistent training, and equals hyperparameter-scaled initialization noise under interpolation. A finite-sample falsification test and seed-swept experiments confirm the theory.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 22 references that are in the map. First posted on arXiv 10 Jun 2026.

Builds on

18 of 22 references.

References · 18

  1. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
  2. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 913
  3. 2017Deep Bayesian Active Learning with Image Datacited by 819
  4. 2018Predictive Uncertainty Estimation via Prior Networkscited by 557
  5. 2019BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learningcited by 432
  6. 2019Deep Evidential Regressioncited by 355
  7. 2018Randomized Prior Functions for Deep Reinforcement Learningcited by 219
  8. 2018Understanding Measures of Uncertainty for Adversarial Example Detectioncited by 177
  9. 2022Quantifying Aleatoric and Epistemic Uncertainty in Machine Learning: Are Conditional Entropy and Mutual Information Appropriate Measures?cited by 125
  10. 2022Pitfalls of Epistemic Uncertainty Quantification through Loss Minimisationcited by 66
  11. 2023Prediction-Oriented Bayesian Active Learningcited by 62
  12. 2023Uncertainty in Natural Language Generation: From Theory to Applicationscited by 61
6 more
  1. 2020Minimum Excess Risk in Bayesian Learningcited by 35
  2. 2023Sources of Uncertainty in Supervised Machine Learning -- A Statisticians' Viewcited by 28
  3. 2019Galaxy Zoo: Probabilistic Morphology through Bayesian CNNs and Active Learningcited by 22
  4. 2025Query-Level Uncertainty in Large Language Modelscited by 14
  5. 2021On out-of-distribution detection with Bayesian neural networkscited by 6
  6. 2026On the Epistemic Uncertainty of Overparametrized Neural Networkscited by 2

Built on by

Not cited yet.