mlsanityData to 7 Oct 2026

A Survey of Uncertainty in Deep Neural Networks

arXiv 2107.03342 ↗First posted 7 Jul 2021cs.LG stat.MLcited by 675

Not on the map. Why?

Abstract (arXiv)

Due to their increasing spread, confidence in neural network predictions became more and more important. However, basic neural networks do not deliver certainty estimates or suffer from over or under confidence. Many researchers have been working on understanding and quantifying uncertainty in a neural network's prediction. As a result, different types and sources of uncertainty have been identified and a variety of approaches to measure and quantify uncertainty in neural networks have been proposed. This work gives a comprehensive overview of uncertainty estimation in neural networks, reviews recent advances in the field, highlights current challenges, and identifies potential research opportunities. It is intended to give anyone interested in uncertainty estimation in neural networks a broad overview and introduction, without presupposing prior knowledge in this field. A comprehensive introduction to the most crucial sources of uncertainty is given and their separation into reducible model uncertainty and not reducible data uncertainty is presented. The modeling of these uncertainties based on deterministic neural networks, Bayesian neural networks, ensemble of neural networks, and test-time data augmentation approaches is introduced and different branches of these fields as well as the latest developments are discussed. For a practical application, we discuss different measures of uncertainty, approaches for the calibration of neural networks and give an overview of existing baselines and implementations. Different examples from the wide spectrum of challenges in different fields give an idea of the needs and challenges regarding uncertainties in practical applications. Additionally, the practical limitations of current methods for mission- and safety-critical real world applications are discussed and an outlook on the next steps towards a broader usage of such methods is given.

Key facts. Cited by 675 papers in the map (data through 7 Oct 2026). Builds on 204 of its 356 references that are in the map. First posted on arXiv 7 Jul 2021.

Builds on

204 of 356 references.

References · 204

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  4. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  5. 2017On Calibration of Modern Neural Networkscited by 4,860
  6. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  7. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  8. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  9. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  10. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  11. 2015Variational Inference with Normalizing Flowscited by 2,161
  12. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
38 more
  1. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  2. 2018Relational inductive biases, deep learning, and graph networkscited by 1,341
  3. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  4. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  5. 2019When Does Label Smoothing Help?cited by 985
  6. 2018Deep Anomaly Detection with Outlier Exposurecited by 939
  7. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 903
  8. 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 813
  9. 2017Deep Bayesian Active Learning with Image Datacited by 812
  10. 2015Optimizing Neural Networks with Kronecker-factored Approximate Curvaturecited by 776
  11. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
  12. 2018Evidential Deep Learning to Quantify Classification Uncertaintycited by 728
  13. 2018Predictive Uncertainty Estimation via Prior Networkscited by 555
  14. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 555
  15. 2015Weight Uncertainty in Neural Networkscited by 554
  16. 2016Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networkscited by 541
  17. 2015Variational Dropout and the Local Reparameterization Trickcited by 489
  18. 2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
  19. 2017Regularizing Neural Networks by Penalizing Confident Output Distributionscited by 461
  20. 2019Likelihood Ratios for Out-of-Distribution Detectioncited by 451
  21. 2019Measuring Calibration in Deep Learningcited by 443
  22. 2019Deep Ensembles: A Loss Landscape Perspectivecited by 438
  23. 2019BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learningcited by 428
  24. 2018Accurate Uncertainties for Deep Learning Using Calibrated Regressioncited by 417
  25. 2020Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Datacited by 408
  26. 2017Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samplescited by 389
  27. 2018Pyro: Deep Universal Probabilistic Programmingcited by 379
  28. 2020BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learningcited by 368
  29. 2019Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibrationcited by 359
  30. 2019Deep Evidential Regressioncited by 354
  31. 2017Snapshot Ensembles: Train 1, get M for freecited by 349
  32. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 347
  33. 2017Variational Continual Learningcited by 335
  34. 2018Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problemcited by 323
  35. 2019On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networkscited by 304
  36. 2015Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networkscited by 295
  37. 2018Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networkscited by 278
  38. 2017Concrete Dropoutcited by 227

50 most cited of 204 All 204 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 17; 2022: 87; 2023: 146; 2024: 185; 2025: 137; 2026: 103.

675 papers.

Sort:

By area:

  1. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
  2. 2022TruFor: Leveraging all-round clues for trustworthy image forgery detection and localizationcited by 130
  3. 2024Benchmarking LLMs via Uncertainty Quantificationcited by 124
  4. 2022RAMBO-RL: Robust Adversarial Model-Based Offline Reinforcement Learningcited by 115
  5. 2024Approaching Human-Level Forecasting with Language Modelscited by 102
  6. 2023Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysiscited by 99
  7. 2023Generative Time Series Forecasting with Diffusion, Denoise, and Disentanglementcited by 89
  8. 2022Source-Free Unsupervised Domain Adaptation: A Surveycited by 79
  9. 2023A Survey on Uncertainty Quantification Methods for Deep Learningcited by 78
  10. 2023Machine learning with data assimilation and uncertainty quantification for dynamical systems: a reviewcited by 74
  11. 2024Value-Incentivized Preference Optimization: A Unified Approach to Online and Offline RLHFcited by 73
  12. 2023Calibration in Deep Learning: A Survey of the State-of-the-Artcited by 72
38 more
  1. 2024VL-Uncertainty: Detecting Hallucination in Large Vision-Language Model via Uncertainty Estimationcited by 71
  2. 2022Trustworthy clinical AI solutions: a unified review of uncertainty quantification in deep learning models for medical image analysiscited by 62
  3. 2023Uncertainty in Natural Language Generation: From Theory to Applicationscited by 61
  4. 2024Confidence Regulation Neurons in Language Modelscited by 57
  5. 2025GAM-Agent: Game-Theoretic and Uncertainty-Aware Collaboration for Complex Visual Reasoningcited by 56
  6. 2023Uncertainty-Penalized Reinforcement Learning from Human Feedback with Diverse Reward LoRA Ensemblescited by 55
  7. 2025Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinationscited by 48
  8. 2023Uncertainty in Natural Language Processing: Sources, Quantification, and Applicationscited by 44
  9. 2024When to Trust LLMs: Aligning Confidence with Response Qualitycited by 42
  10. 2023Disinformation Detection: An Evolving Challenge in the Age of LLMscited by 41
  11. 2022A Survey of Active Learning for Natural Language Processingcited by 40
  12. 2023Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Modelscited by 40
  13. 2022A Review of Uncertainty for Deep Reinforcement Learningcited by 36
  14. 2025UProp: Investigating the Uncertainty Propagation of LLMs in Multi-Step Agentic Decision-Makingcited by 36
  15. 2022Uncertainty Estimation for Language Reward Modelscited by 35
  16. 2024Uncertainty quantification in fine-tuned LLMs using LoRA ensemblescited by 35
  17. 2024A Stitch in Time Saves Nine: Small VLM is a Precise Guidance for Accelerating Large VLMscited by 35
  18. 2024Improving Uncertainty Estimation through Semantically Diverse Language Generationcited by 34
  19. 2022Predictive Inference with Feature Conformal Predictioncited by 33
  20. 2024Active Evaluation Acquisition for Efficient LLM Benchmarkingcited by 33
  21. 2022Discovering and forecasting extreme events via active learning in neural operatorscited by 32
  22. 2023Topology-Aware Uncertainty for Image Segmentationcited by 32
  23. 2024On the Challenges and Opportunities in Generative AIcited by 31
  24. 2022Uncertainty-Aware Deep Multi-View Photometric Stereocited by 30
  25. 2022Behavioral Intention Prediction in Driving Scenes: A Surveycited by 30
  26. 2023Single-model uncertainty quantification in neural network potentials does not consistently outperform model ensemblescited by 30
  27. 2024Review of multimodal machine learning approaches in healthcarecited by 30
  28. 2022A review of predictive uncertainty estimation with machine learningcited by 29
  29. 2023Learning to Intervene on Concept Bottleneckscited by 29
  30. 2023UnCRtainTS: Uncertainty Quantification for Cloud Removal in Optical Satellite Time Seriescited by 28
  31. 2023Quantification of Uncertainty with Adversarial Modelscited by 28
  32. 2024Uncertainty-Aware Evaluation for Vision-Language Modelscited by 28
  33. 2023A Large-Scale Study of Probabilistic Calibration in Neural Network Regressioncited by 26
  34. 2023A Primer on Bayesian Neural Networks: Review and Debatescited by 26
  35. 2024CreINNs: Credal-Set Interval Neural Networks for Uncertainty Estimation in Classification Taskscited by 26
  36. 2021Temperature as Uncertainty in Contrastive Learningcited by 25
  37. 2022On Calibrating Semantic Segmentation Models: Analyses and An Algorithmcited by 25
  38. 2023ELFNet: Evidential Local-global Fusion for Stereo Matchingcited by 25

50 most cited of 675 All 675 (JSON)