mlsanityData to 7 Oct 2026

Pitfalls of Epistemic Uncertainty Quantification through Loss Minimisation

arXiv 2203.06102 ↗First posted 11 Mar 2022cs.LG stat.MLcited by 65

Not on the map. Why?

Abstract (arXiv)

Uncertainty quantification has received increasing attention in machine learning in the recent past. In particular, a distinction between aleatoric and epistemic uncertainty has been found useful in this regard. The latter refers to the learner's (lack of) knowledge and appears to be especially difficult to measure and quantify. In this paper, we analyse a recent proposal based on the idea of a second-order learner, which yields predictions in the form of distributions over probability distributions. While standard (first-order) learners can be trained to predict accurate probabilities, namely by minimising suitable loss functions on sample data, we show that loss minimisation does not work for second-order predictors: The loss functions proposed for inducing such predictors do not incentivise the learner to represent its epistemic uncertainty in a faithful way.

Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). Builds on 14 of its 35 references that are in the map. First posted on arXiv 11 Mar 2022.

Builds on

14 of 35 references.

References · 14

  1. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  2. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 903
  3. 2018Evidential Deep Learning to Quantify Classification Uncertaintycited by 728
  4. 2018Predictive Uncertainty Estimation via Prior Networkscited by 555
  5. 2013A General Framework for Updating Belief Distributionscited by 172
  6. 2020Posterior Network: Uncertainty Estimation without OOD Samples via Density-Based Pseudo-Countscited by 163
  7. 2019Ensemble Distribution Distillationcited by 158
  8. 2019Reverse KL-Divergence Training of Prior Networks: Improved Uncertainty and Adversarial Robustnesscited by 104
  9. 2021DEUP: Direct Epistemic Uncertainty Predictioncited by 102
  10. 2016On the Safety of Machine Learning: Cyber-Physical Systems, Decision Sciences, and Data Productscited by 52
  11. 2020Aleatoric and Epistemic Uncertainty with Random Forestscited by 42
  12. 2020Evaluating Robustness of Predictive Uncertainty Estimation: Are Dirichlet-based Models Reliable?cited by 39
2 more
  1. 2016Engineering Safety in Machine Learningcited by 24
  2. 2017Concave losses for robust dictionary learningcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 9; 2024: 17; 2025: 20; 2026: 18.

65 papers.

Sort:

By area:

  1. 2023Uncertainty Estimation by Fisher Information-based Evidential Deep Learningcited by 58
  2. 2024Estimating Epistemic and Aleatoric Uncertainty with a Single Modelcited by 52
  3. 2022The Unreasonable Effectiveness of Deep Evidential Regressioncited by 40
  4. 2023Is the Volume of a Credal Set a Good Measure for Epistemic Uncertainty?cited by 40
  5. 2023On Second-Order Scoring Rules for Epistemic Uncertainty Quantificationcited by 38
  6. 2023Second-Order Uncertainty Quantification: A Distance-Based Approachcited by 35
  7. 2024Is Epistemic Uncertainty Faithfully Represented by Evidential Deep Learning Methods?cited by 34
  8. 2025Position: Uncertainty Quantification Needs Reassessment for Large-language Model Agentscited by 27
  9. 2024A Comprehensive Survey on Evidential Deep Learning and Its Applicationscited by 25
  10. 2024Conformalized Credal Set Predictorscited by 21
  11. 2024Quantifying Aleatoric and Epistemic Uncertainty with Proper Scoring Rulescited by 21
  12. 2023Random-Set Neural Networks (RS-NN)cited by 19
38 more
  1. 2024Conformalized Credal Regions for Classification with Ambiguous Ground Truthcited by 19
  2. 2024Experts Don't Cheat: Learning What You Don't Know By Predicting Pairscited by 18
  3. 2024Are Uncertainty Quantification Capabilities of Evidential Deep Learning a Mirage?cited by 17
  4. 2024(Implicit) Ensembles of Ensembles: Epistemic Uncertainty Collapse in Large Modelscited by 15
  5. 2024Revisiting Essential and Nonessential Settings of Evidential Deep Learningcited by 15
  6. 2023Robust Statistical Comparison of Random Variables with Locally Varying Scale of Measurementcited by 13
  7. 2024Hyper Evidential Deep Learning to Quantify Composite Classification Uncertaintycited by 10
  8. 2025Uncertainty Estimation by Flexible Evidential Deep Learningcited by 10
  9. 2023Proper Scoring Rules for Survival Analysiscited by 9
  10. 2025An Axiomatic Assessment of Entropy- and Variance-based Uncertainty Quantification in Regressioncited by 9
  11. 2023Dirichlet-based Uncertainty Quantification for Personalized Federated Learning with Improved Posterior Networkscited by 8
  12. 2024Provable Uncertainty Decomposition via Higher-Order Calibrationcited by 8
  13. 2025Conformal Prediction and Human Decision Makingcited by 8
  14. 2025Epistemic Artificial Intelligence is Essential for Machine Learning Models to Truly 'Know When They Do Not Know'cited by 8
  15. 2025Credal and Interval Deep Evidential Classificationscited by 8
  16. 2025A calibration test for evaluating set-based epistemic uncertainty representationscited by 7
  17. 2025Reducing Aleatoric and Epistemic Uncertainty through Multi-modal Data Acquisitioncited by 5
  18. 2025Revisiting Active Learning under (Human) Label Variationcited by 4
  19. 2026ELVAE: Evidential Learning-Based Variational Autoencoder for Uncertainty-Aware Generationcited by 3
  20. 2024Uncertainty-aware Evaluation of Auxiliary Anomalies with the Expected Anomaly Posteriorcited by 2
  21. 2024Predictive Uncertainty Quantification for Bird's Eye View Segmentation: A Benchmark and Novel Loss Functioncited by 2
  22. 2024Linear Opinion Pooling for Uncertainty Quantification on Graphscited by 2
  23. 2024On the Calibration of Epistemic Uncertainty: Principles, Paradoxes and Conflictual Losscited by 2
  24. 2025Rethinking Approximate Gaussian Inference in Classificationcited by 2
  25. 2025Generalized Regularized Evidential Deep Learning Models: Theory and Comprehensive Evaluationcited by 2
  26. 2025Disentangling Uncertainties by Learning Compressed Data Representationcited by 1
  27. 2025Uncertainty Quantification with Proper Scoring Rules: Adjusting Measures to Prediction Taskscited by 1
  28. 2025On Uncertainty Calibration for Equivariant Functionscited by 1
  29. 2025Maxitive Donsker-Varadhan Formulation for Possibilistic Variational Inferencecited by 1
  30. 2026Rethinking Uncertainty Quantification and Entanglement in Image Segmentationcited by 1
  31. 2026Generalized Evidential Deep Learning: From a Bayesian Perspectivecited by 1
  32. 2026Decision-Aligned Evaluation of Uncertainty Quantificationcited by 1
  33. 2023Variational Inference on the Final-Layer Output of Neural Networkscited by 0
  34. 2024CUQ-GNN: Committee-based Graph Uncertainty Quantification using Posterior Networkscited by 0
  35. 2025Evidential Uncertainty Probes for Graph Neural Networkscited by 0
  36. 2025Detecting underdetermination in parameterized quantum circuitscited by 0
  37. 2025Principled Input-Output-Conditioned Post-Hoc Uncertainty Estimation for Regression Networkscited by 0
  38. 2025Explainable Evidential Clusteringcited by 0

50 most cited of 65 All 65 (JSON)