mlsanityData to 14 Oct 2026

Quantifying Epistemic Uncertainty in Deep Learning

arXiv 2110.12122 ↗First posted 23 Oct 2021cs.LG stat.ME stat.MLcited by 10

Not on the map. Why?

Abstract (arXiv)

Uncertainty quantification is at the core of the reliability and robustness of machine learning. In this paper, we provide a theoretical framework to dissect the uncertainty, especially the textitepistemic component, in deep learning into textitprocedural variability (from the training procedure) and textitdata variability (from the training data), which is the first such attempt in the literature to our best knowledge. We then propose two approaches to estimate these uncertainties, one based on influence function and one on batching. We demonstrate how our approaches overcome the computational difficulties in applying classical statistical methods. Experimental evaluations on multiple problem settings corroborate our theory and illustrate how our framework and estimation can provide direct guidance on modeling and data collection efforts.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 47 of its 96 references that are in the map. First posted on arXiv 23 Oct 2021.

Builds on

47 of 96 references.

References · 47

  1. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  2. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  3. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  4. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  5. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
  6. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
  7. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 913
  8. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 787
  9. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 738
  10. 2016Distribution-Free Predictive Inference For Regressioncited by 668
  11. 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 659
  12. 2019Conformalized Quantile Regressioncited by 629
35 more
  1. 2018Gradient Descent Finds Global Minima of Deep Neural Networkscited by 574
  2. 2019On Exact Computation with an Infinitely Wide Neural Netcited by 573
  3. 2019Deep Ensembles: A Loss Landscape Perspectivecited by 443
  4. 2018Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layerscited by 340
  5. 2017The Expressive Power of Neural Networks: A View from the Widthcited by 320
  6. 2015Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networkscited by 295
  7. 2018Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Datacited by 261
  8. 2020Influence Functions in Deep Learning Are Fragilecited by 252
  9. 2019Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networkscited by 222
  10. 2019Predictive inference with the jackknife+cited by 211
  11. 2020Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learningcited by 210
  12. 2019Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivationcited by 149
  13. 2019On the Accuracy of Influence Functions for Measuring Group Effectscited by 147
  14. 2020Tensor Programs II: Neural Tangent Kernel for Any Architecturecited by 143
  15. 2018Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networkscited by 141
  16. 2015Why M Heads are Better than One: Training a Diverse Ensemble of Deep Networkscited by 139
  17. 2020Neural Thompson Samplingcited by 128
  18. 2017Approximating Continuous Functions by ReLU Nets of Minimal Widthcited by 122
  19. 2018High-Quality Prediction Intervals for Deep Learning: A Distribution-Free, Ensembled Approachcited by 117
  20. 2020Bayesian Deep Ensembles via the Neural Tangent Kernelcited by 90
  21. 2013A Conformal Prediction Approach to Explore Functional Datacited by 86
  22. 2018On the Convergence Rate of Training Recurrent Neural Networkscited by 82
  23. 2019Enhanced Convolutional Neural Tangent Kernelscited by 76
  24. 2013Divide and Conquer Kernel Ridge Regression: A Distributed Algorithm with Minimax Optimal Ratescited by 65
  25. 2019Generalization Error Bounds of Gradient Descent for Learning Over-parameterized Deep ReLU Networkscited by 60
  26. 2018Evaluating Uncertainty Quantification in End-to-End Autonomous Driving Controlcited by 55
  27. 2019A type of generalization error induced by initialization in deep neural networkscited by 33
  28. 2018Gaussian Process Conditional Density Estimationcited by 23
  29. 2019Conditional Density Estimation Tools in Python and R with Applications to Photometric Redshifts and Likelihood-Free Cosmological Inferencecited by 21
  30. 2016Nonparametric Conditional Density Estimation in a High-Dimensional Regression Settingcited by 20
  31. 2020Frequentist Uncertainty in Recurrent Neural Networks via Blockwise Influence Functionscited by 18
  32. 2021Learning Prediction Intervals for Regression: Generalization and Calibrationcited by 16
  33. 2021Understanding the Under-Coverage Bias in Uncertainty Estimationcited by 10
  34. 2017Discriminative Learning of Prediction Intervalscited by 5
  35. 2019HDI-Forest: Highest Density Interval Regression Forestcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 1; 2024: 2; 2025: 2; 2026: 2.

10 papers.

  1. 2025Plan and Budget: Effective and Efficient Test-Time Scaling on Reasoning Large Language Modelscited by 32
  2. 2023Efficient Uncertainty Quantification and Reduction for Over-Parameterized Neural Networkscited by 10
  3. 2022Improving Deep Neural Network Random Initialization Through Neuronal Rewiringcited by 7
  4. 2022The Infinitesimal Jackknife and Combinations of Modelscited by 5
  5. 2024Uncertainty Quantification on Graph Learning: A Surveycited by 4
  6. 2024Generative vs. Discriminative modeling under the lens of uncertainty quantificationcited by 2
  7. 2026Credal Large Language Models for Semantic Commitment under Uncertaintycited by 2
  8. 2026Subjective Risk Decomposition: A New View for Uncertainty Quantificationcited by 1
  9. 2022Model error and its estimation, with particular application to loss reservingcited by 0
  10. 2025Foundations of Artificial Intelligence Frameworks: Notion and Limits of AGIcited by 0