mlsanityData to 7 Oct 2026

A Unified Evaluation Framework for Epistemic Predictions

arXiv 2501.16912 ↗First posted 28 Jan 2025cs.LGcited by 10

Not on the map. Why?

Abstract (arXiv)

Predictions of uncertainty-aware models are diverse, ranging from single point estimates (often averaged over prediction samples) to predictive distributions, to set-valued or credal-set representations. We propose a novel unified evaluation framework for uncertainty-aware classifiers, applicable to a wide range of model classes, which allows users to tailor the trade-off between accuracy and precision of predictions via a suitably designed performance metric. This makes possible the selection of the most suitable model for a particular real-world application as a function of the desired trade-off. Our experiments, concerning Bayesian, ensemble, evidential, deterministic, credal and belief function classifiers on the CIFAR-10, MNIST and CIFAR-100 datasets, show that the metric behaves as desired.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 16 of its 87 references that are in the map. First posted on arXiv 28 Jan 2025.

Builds on

16 of 87 references.

References · 16

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  3. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  4. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 903
  5. 2018Evidential Deep Learning to Quantify Classification Uncertaintycited by 728
  6. 2020Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Userscited by 336
  7. 2021Deep Deterministic Uncertainty: A Simple Baselinecited by 200
  8. 2020Bayesian Neural Networks: An Introduction and Surveycited by 130
  9. 2023Credal Bayesian Deep Learningcited by 45
  10. 2020Fast Predictive Uncertainty for Classification with Bayesian Deep Networkscited by 31
  11. 2024Credal Learning Theorycited by 30
  12. 2024CreINNs: Credal-Set Interval Neural Networks for Uncertainty Estimation in Classification Taskscited by 26
4 more
  1. 2023Random-Set Neural Networks (RS-NN)cited by 19
  2. 2021An evidential classifier based on Dempster-Shafer theory and deep learningcited by 18
  3. 2021Ensemble Quantile Networks: Uncertainty-Aware Reinforcement Learning with Applications in Autonomous Drivingcited by 17
  4. 2022Epistemic Deep Learningcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 8.

10 papers.

  1. 2025Epistemic Artificial Intelligence is Essential for Machine Learning Models to Truly 'Know When They Do Not Know'cited by 8
  2. 2026Learning Credal Ensembles via Distributionally Robust Optimizationcited by 6
  3. 2025Epistemic Wrapping for Uncertainty Quantificationcited by 5
  4. 2026Random-Set Graph Neural Networkscited by 3
  5. 2026Statistical inference with belief functions: A surveycited by 2
  6. 2026Credal Large Language Models for Semantic Commitment under Uncertaintycited by 2
  7. 2026Set-based v.s. Distribution-based Representations of Epistemic Uncertainty: A Comparative Studycited by 1
  8. 2026A formal definition and meta-model for a machine theory of mindcited by 1
  9. 2026What Intermediate Layers Know: Detecting Jailbreaks from Entropy Dynamicscited by 1
  10. 2026XU-RS: Explaining Credal Width in Random-Set Language Modelscited by 0