mlsanityData to 7 Oct 2026

Exploring the Limits of Epistemic Uncertainty Quantification in Low-Shot Settings

arXiv 2111.09808 ↗First posted 18 Nov 2021cs.LG cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Uncertainty quantification in neural network promises to increase safety of AI systems, but it is not clear how performance might vary with the training set size. In this paper we evaluate seven uncertainty methods on Fashion MNIST and CIFAR10, as we sub-sample and produce varied training set sizes. We find that calibration error and out of distribution detection performance strongly depend on the training set size, with most methods being miscalibrated on the test set with small training sets. Gradient-based methods seem to poorly estimate epistemic uncertainty and are the most affected by training set size. We expect our results can guide future research into uncertainty quantification and help practitioners select methods based on their particular available data.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 13 of its 20 references that are in the map. First posted on arXiv 18 Nov 2021.

Builds on

13 of 20 references.

References · 13

  1. 2017On Calibration of Modern Neural Networkscited by 4,860
  2. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  3. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  4. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  5. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  6. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 903
  7. 2021A Survey of Uncertainty in Deep Neural Networkscited by 675
  8. 2018Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batchescited by 133
  9. 2020A Review and Comparative Study on Probabilistic Object Detection in Autonomous Drivingcited by 115
  10. 2018The Limits and Potentials of Deep Learning for Roboticscited by 104
  11. 2019DropConnect Is Effective in Modeling Uncertainty of Bayesian Deep Networkscited by 92
  12. 2020The Case for Bayesian Deep Learningcited by 68
1 more
  1. 2018Classification Uncertainty of Deep Neural Networks Based on Gradient Informationcited by 43

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 1; 2025: 2.

4 papers.

  1. 2023Uncertainty Quantification in Machine Learning for Biosignal Applications -- A Reviewcited by 7
  2. 2024Measuring Orthogonality as the Blind-Spot of Uncertainty Disentanglementcited by 4
  3. 2025Unreliable Uncertainty Estimates with Monte Carlo Dropoutcited by 2
  4. 2025Modeling of AUV Dynamics with Limited Resources: Efficient Online Learning Using Uncertaintycited by 1