# Quantifying Aleatoric and Epistemic Uncertainty in Machine Learning: Are Conditional Entropy and Mutual Information Appropriate Measures?

arXiv: https://arxiv.org/abs/2209.03302 · HTML: https://mlsanity.com/paper/2209.03302/

## Key facts

- Cited by 125 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 59 references that are in the map.
- First posted on arXiv 7 Sep 2022.

## Builds on (most cited first, 29 of 29)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [Predictive Uncertainty Estimation via Prior Networks](https://mlsanity.com/paper/1802.10501/) (2018)
- [Laplace Redux -- Effortless Bayesian Deep Learning](https://mlsanity.com/paper/2106.14806/) (2021)
- [Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning](https://mlsanity.com/paper/2002.06470/) (2020)
- [Understanding Measures of Uncertainty for Adversarial Example Detection](https://mlsanity.com/paper/1803.08533/) (2018)
- [Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons](https://mlsanity.com/paper/2201.07766/) (2022)
- [Posterior Network: Uncertainty Estimation without OOD Samples via Density-Based Pseudo-Counts](https://mlsanity.com/paper/2006.09239/) (2020)
- [Ensemble Distribution Distillation](https://mlsanity.com/paper/1905.00076/) (2019)
- [SUNRISE: A Simple Unified Framework for Ensemble Learning in Deep Reinforcement Learning](https://mlsanity.com/paper/2007.04938/) (2020)
- [A Survey on Active Learning and Human-in-the-Loop Deep Learning for Medical Image Analysis](https://mlsanity.com/paper/1910.02923/) (2019)
- [Reverse KL-Divergence Training of Prior Networks: Improved Uncertainty and Adversarial Robustness](https://mlsanity.com/paper/1905.13472/) (2019)
- [DropConnect Is Effective in Modeling Uncertainty of Bayesian Deep Networks](https://mlsanity.com/paper/1906.04569/) (2019)
- [The Case for Bayesian Deep Learning](https://mlsanity.com/paper/2001.10995/) (2020)
- [Deep Ensembles Work, But Are They Necessary?](https://mlsanity.com/paper/2202.06985/) (2022)
- [Evaluating Uncertainty Quantification in End-to-End Autonomous Driving Control](https://mlsanity.com/paper/1811.06817/) (2018)
- [Evaluating Robustness of Predictive Uncertainty Estimation: Are Dirichlet-based Models Reliable?](https://mlsanity.com/paper/2010.14986/) (2020)
- [Active Learning for Sequence Tagging with Deep Pre-trained Models and Bayesian Uncertainty Estimates](https://mlsanity.com/paper/2101.08133/) (2021)
- [Disentangling Epistemic and Aleatoric Uncertainty in Reinforcement Learning](https://mlsanity.com/paper/2206.01558/) (2022)
- [FiLM-Ensemble: Probabilistic Deep Learning via Feature-wise Linear Modulation](https://mlsanity.com/paper/2206.00050/) (2022)
- [Being a Bit Frequentist Improves Bayesian Neural Networks](https://mlsanity.com/paper/2106.10065/) (2021)
- [Decomposition of Uncertainty in Bayesian Deep Learning for Efficient and Risk-sensitive Learning](https://mlsanity.com/paper/1710.07283/) (2017)
- [Analytic Mutual Information in Bayesian Neural Networks](https://mlsanity.com/paper/2201.09815/) (2022)
- [Marginal and Joint Cross-Entropies & Predictives for Online Bayesian Inference, Active Learning, and Active Sampling](https://mlsanity.com/paper/2205.08766/) (2022)

## Built on by (most cited first, 30 of 125)

- [Benchmarking Uncertainty Disentanglement: Specialized Uncertainties for Specialized Tasks](https://mlsanity.com/paper/2402.19460/) (2024)
- [Do LLMs Know When to NOT Answer? Investigating Abstention Abilities of Large Language Models](https://mlsanity.com/paper/2407.16221/) (2024)
- [A Survey of Uncertainty Estimation in LLMs: Theory Meets Practice](https://mlsanity.com/paper/2410.15326/) (2024)
- [Uncertainty Quantification for In-Context Learning of Large Language Models](https://mlsanity.com/paper/2402.10189/) (2024)
- [Rethinking Aleatoric and Epistemic Uncertainty](https://mlsanity.com/paper/2412.20892/) (2024)
- [Is the Volume of a Credal Set a Good Measure for Epistemic Uncertainty?](https://mlsanity.com/paper/2306.09586/) (2023)
- [Uncertainty quantification in fine-tuned LLMs using LoRA ensembles](https://mlsanity.com/paper/2402.12264/) (2024)
- [Explaining Predictive Uncertainty with Information Theoretic Shapley Values](https://mlsanity.com/paper/2306.05724/) (2023)
- [Second-Order Uncertainty Quantification: A Distance-Based Approach](https://mlsanity.com/paper/2312.00995/) (2023)
- [Is Epistemic Uncertainty Faithfully Represented by Evidential Deep Learning Methods?](https://mlsanity.com/paper/2402.09056/) (2024)
- [Active Evaluation Acquisition for Efficient LLM Benchmarking](https://mlsanity.com/paper/2410.05952/) (2024)
- [Position: Uncertainty Quantification Needs Reassessment for Large-language Model Agents](https://mlsanity.com/paper/2505.22655/) (2025)
- [On the Challenges and Opportunities in Generative AI](https://mlsanity.com/paper/2403.00025/) (2024)
- [Introducing an Improved Information-Theoretic Measure of Predictive Uncertainty](https://mlsanity.com/paper/2311.08309/) (2023)
- [From Risk to Uncertainty: Generating Predictive Uncertainty Measures via Bayesian Estimation](https://mlsanity.com/paper/2402.10727/) (2024)
- [A Comprehensive Survey on Evidential Deep Learning and Its Applications](https://mlsanity.com/paper/2409.04720/) (2024)
- [Label-wise Aleatoric and Epistemic Uncertainty Quantification](https://mlsanity.com/paper/2406.02354/) (2024)
- [Quantifying Aleatoric and Epistemic Uncertainty with Proper Scoring Rules](https://mlsanity.com/paper/2404.12215/) (2024)
- [Second-Order Uncertainty Quantification: Variance-Based Measures](https://mlsanity.com/paper/2401.00276/) (2023)
- [Conformalized Credal Regions for Classification with Ambiguous Ground Truth](https://mlsanity.com/paper/2411.04852/) (2024)
- [The Illusion of Certainty: Uncertainty Quantification for LLMs Fails under Ambiguity](https://mlsanity.com/paper/2511.04418/) (2025)
- [On Information-Theoretic Measures of Predictive Uncertainty](https://mlsanity.com/paper/2410.10786/) (2024)
- [(Implicit) Ensembles of Ensembles: Epistemic Uncertainty Collapse in Large Models](https://mlsanity.com/paper/2409.02628/) (2024)
- [Knowing When to Abstain: Medical LLMs Under Clinical Uncertainty](https://mlsanity.com/paper/2601.12471/) (2026)
- [A Novel Bayes' Theorem for Upper Probabilities](https://mlsanity.com/paper/2307.06831/) (2023)
- [Scalable Bayesian Learning with posteriors](https://mlsanity.com/paper/2406.00104/) (2024)
- [Integral Imprecise Probability Metrics](https://mlsanity.com/paper/2505.16156/) (2025)
- [Pretrained Visual Uncertainties](https://mlsanity.com/paper/2402.16569/) (2024)
- [A Structured Review of Literature on Uncertainty in Machine Learning & Deep Learning](https://mlsanity.com/paper/2406.00332/) (2024)
- [Grammars of Formal Uncertainty: When to Trust LLMs in Automated Reasoning Tasks](https://mlsanity.com/paper/2505.20047/) (2025)

## Abstract (arXiv)

The quantification of aleatoric and epistemic uncertainty in terms of conditional entropy and mutual information, respectively, has recently become quite common in machine learning. While the properties of these measures, which are rooted in information theory, seem appealing at first glance, we identify various incoherencies that call their appropriateness into question. In addition to the measures themselves, we critically discuss the idea of an additive decomposition of total uncertainty into its aleatoric and epistemic constituents. Experiments across different computer vision tasks support our theoretical findings and raise concerns about current practice in uncertainty quantification.