mlsanityData to 14 Oct 2026

Quantifying Aleatoric and Epistemic Uncertainty with Proper Scoring Rules

arXiv 2404.12215 ↗First posted 18 Apr 2024cs.LG stat.MLcited by 21

Not on the map. Why?

Abstract (arXiv)

Uncertainty representation and quantification are paramount in machine learning and constitute an important prerequisite for safety-critical applications. In this paper, we propose novel measures for the quantification of aleatoric and epistemic uncertainty based on proper scoring rules, which are loss functions with the meaningful property that they incentivize the learner to predict ground-truth (conditional) probabilities. We assume two common representations of (epistemic) uncertainty, namely, in terms of a credal set, i.e. a set of probability distributions, or a second-order distribution, i.e., a distribution over probability distributions. Our framework establishes a natural bridge between these representations. We provide a formal justification of our approach and introduce new measures of epistemic and aleatoric uncertainty as concrete instantiations.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Builds on 10 of its 33 references that are in the map. First posted on arXiv 18 Apr 2024.

Builds on

10 of 33 references.

References · 10

  1. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
  2. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 913
  3. 2018Understanding Measures of Uncertainty for Adversarial Example Detectioncited by 177
  4. 2022Quantifying Aleatoric and Epistemic Uncertainty in Machine Learning: Are Conditional Entropy and Mutual Information Appropriate Measures?cited by 125
  5. 2019DropConnect Is Effective in Modeling Uncertainty of Bayesian Deep Networkscited by 92
  6. 2022Pitfalls of Epistemic Uncertainty Quantification through Loss Minimisationcited by 66
  7. 2022On Uncertainty, Tempering, and Data Augmentation in Bayesian Classificationcited by 46
  8. 2023Introducing an Improved Information-Theoretic Measure of Predictive Uncertaintycited by 28
  9. 2024From Risk to Uncertainty: Generating Predictive Uncertainty Measures via Bayesian Estimationcited by 28
  10. 2023Second-Order Uncertainty Quantification: Variance-Based Measurescited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 8; 2026: 7.

21 papers.

Sort:

By area:

  1. 2024Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraphcited by 105
  2. 2024Rethinking Aleatoric and Epistemic Uncertaintycited by 42
  3. 2024Rethinking Uncertainty Estimation in LLMs: A Principled Single-Sequence Measurecited by 18
  4. 2024On Information-Theoretic Measures of Predictive Uncertaintycited by 16
  5. 2024Scalable Bayesian Learning with posteriorscited by 13
  6. 2024Provable Uncertainty Decomposition via Higher-Order Calibrationcited by 9
  7. 2025Position: Epistemic uncertainty estimation methods are fundamentally incompletecited by 9
  8. 2025CLEAR: Calibrated Learning for Epistemic and Aleatoric Riskcited by 6
  9. 2025Hallucination Detection on a Budget: Efficient Bayesian Estimation of Semantic Entropycited by 5
  10. 2025Uncertainty Quantification for Regression using Proper Scoring Rulescited by 5
  11. 2025Uncertainty Quantification for Machine Learning: One Size Does Not Fit Allcited by 4
  12. 2025Uncertainty Quantification for Regression: A Unified Framework based on kernel scorescited by 2
9 more
  1. 2026Quantification of Credal Uncertainty: A Distance-Based Approachcited by 2
  2. 2026Evaluating Epistemic Uncertainty: Beyond OOD Detection and Active Learningcited by 2
  3. 2025Multidimensional Uncertainty Quantification via Optimal Transportcited by 1
  4. 2025Epistemic Reject Option Predictioncited by 1
  5. 2026Subjective Risk Decomposition: A New View for Uncertainty Quantificationcited by 1
  6. 2026Beyond Accuracy: Evaluating Posterior Fidelity of Diffusion Inverse Solverscited by 0
  7. 2026The Minimax Rate of Perturbed Second-Order Calibrationcited by 0
  8. 2026The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribingcited by 0
  9. 2026How Much Imprecision is Enough Imprecision in my Classifier? A Practical Elicitation Procedurecited by 0