mlsanityData to 7 Oct 2026

Deep Deterministic Uncertainty: A Simple Baseline

arXiv 2102.11582 ↗First posted 23 Feb 2021cs.LG stat.MLcited by 200

Not on the map. Why?

Abstract (arXiv)

Reliable uncertainty from deterministic single-forward pass models is sought after because conventional methods of uncertainty quantification are computationally expensive. We take two complex single-forward-pass uncertainty approaches, DUQ and SNGP, and examine whether they mainly rely on a well-regularized feature space. Crucially, without using their more complex methods for estimating uncertainty, a single softmax neural net with such a feature-space, achieved via residual connections and spectral normalization, *outperforms* DUQ and SNGP's epistemic uncertainty predictions using simple Gaussian Discriminant Analysis *post-training* as a separate feature-space density estimator -- without fine-tuning on OoD data, feature ensembling, or input pre-procressing. This conceptually simple *Deep Deterministic Uncertainty (DDU)* baseline can also be used to disentangle aleatoric and epistemic uncertainty and performs as well as Deep Ensembles, the state-of-the art for uncertainty prediction, on several OoD benchmarks (CIFAR-10/100 vs SVHN/Tiny-ImageNet, ImageNet vs ImageNet-O) as well as in active learning settings across different model architectures, yet is *computationally cheaper*.

Key facts. Cited by 200 papers in the map (data through 7 Oct 2026). Builds on 49 of its 77 references that are in the map. First posted on arXiv 23 Feb 2021.

Builds on

49 of 77 references.

References · 49

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2016Densely Connected Convolutional Networkscited by 6,691
  5. 2017On Calibration of Modern Neural Networkscited by 4,860
  6. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  7. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  8. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  9. 2013Auto-Encoding Variational Bayescited by 3,980
  10. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  11. 2016Wide Residual Networkscited by 3,057
  12. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
37 more
  1. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  2. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  3. 2017Improved Training of Wasserstein GANscited by 2,223
  4. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  5. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  6. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  7. 2019Natural Adversarial Examplescited by 1,401
  8. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  9. 2020Energy-based Out-of-distribution Detectioncited by 1,296
  10. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  11. 2014NICE: Non-linear Independent Components Estimationcited by 1,052
  12. 2018Deep Anomaly Detection with Outlier Exposurecited by 939
  13. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 916
  14. 2017Deep Bayesian Active Learning with Image Datacited by 812
  15. 2018Predictive Uncertainty Estimation via Prior Networkscited by 555
  16. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 555
  17. 2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
  18. 2019BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learningcited by 428
  19. 2020Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Datacited by 408
  20. 2017Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samplescited by 389
  21. 2020Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awarenesscited by 389
  22. 2020BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learningcited by 368
  23. 2018Regularisation of Neural Networks by Enforcing Lipschitz Continuitycited by 311
  24. 2015Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networkscited by 295
  25. 2018Learning Confidence for Out-of-Distribution Detection in Neural Networkscited by 245
  26. 2020Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networkscited by 227
  27. 2020Contrastive Training for Improved Out-of-Distribution Detectioncited by 186
  28. 2018Understanding Measures of Uncertainty for Adversarial Example Detectioncited by 177
  29. 2020Efficient and Scalable Bayesian Neural Nets with Rank-1 Factorscited by 171
  30. 2019Ensemble Distribution Distillationcited by 158
  31. 2018Evaluating Bayesian Deep Learning Methods for Semantic Segmentationcited by 143
  32. 2020Bayesian Deep Ensembles via the Neural Tangent Kernelcited by 90
  33. 2019A Systematic Comparison of Bayesian Deep Learning Robustness in Diabetic Retinopathy Taskscited by 62
  34. 2020Feature Space Singularity for Out-of-Distribution Detectioncited by 49
  35. 2020Autonomous Driving with Deep Learning: A Survey of State-of-Art Technologiescited by 41
  36. 2020Unpacking Information Bottlenecks: Unifying Information-Theoretic Objectives in Deep Learningcited by 16
  37. 2020The Hidden Uncertainty in a Neural Networks Activationscited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 7; 2023: 34; 2024: 59; 2025: 60; 2026: 37.

200 papers.

Sort:

By area:

  1. 2022Out-of-Distribution Detection and Selective Generation for Conditional Language Modelscited by 193
  2. 2024Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similaritiescited by 156
  3. 2024Benchmarking Uncertainty Disentanglement: Specialized Uncertainties for Specialized Taskscited by 78
  4. 2024Calibrating Large Language Models Using Their Generations Onlycited by 63
  5. 2024A Survey of Uncertainty Estimation in LLMs: Theory Meets Practicecited by 54
  6. 2024Estimating Epistemic and Aleatoric Uncertainty with a Single Modelcited by 52
  7. 2024Rethinking Aleatoric and Epistemic Uncertaintycited by 40
  8. 2021Stochastic Batch Acquisition: A Simple Baseline for Deep Active Learningcited by 36
  9. 2025Mahalanobis++: Improving OOD Detection via Feature Normalizationcited by 30
  10. 2023Quantification of Uncertainty with Adversarial Modelscited by 28
  11. 2023Introducing an Improved Information-Theoretic Measure of Predictive Uncertaintycited by 28
  12. 2024From Risk to Uncertainty: Generating Predictive Uncertainty Measures via Bayesian Estimationcited by 27
38 more
  1. 2025Position: Uncertainty Quantification Needs Reassessment for Large-language Model Agentscited by 27
  2. 2023Beyond Confidence: Reliable Models Should Also Consider Atypicalitycited by 25
  3. 2024ValUES: A Framework for Systematic Validation of Uncertainty Estimation in Semantic Segmentationcited by 25
  4. 2023Continual Learning in Medical Image Analysis: A Comprehensive Review of Recent Advancements and Future Prospectscited by 22
  5. 2024Uncertainty Estimation by Density Aware Evidential Deep Learningcited by 22
  6. 2023On the Variance of Neural Network Training with respect to Test Sets and Distributionscited by 20
  7. 2025Understanding Model Calibration -- A gentle introduction and visual exploration of calibration and the expected calibration error (ECE)cited by 20
  8. 2025Toward Real-world BEV Perception: Depth Uncertainty Estimation via Gaussian Splattingcited by 20
  9. 2023Random-Set Neural Networks (RS-NN)cited by 19
  10. 2024Conformal Semantic Image Segmentation: Post-hoc Quantification of Predictive Uncertaintycited by 19
  11. 2024Deep Bayesian Active Learning for Preference Modeling in Large Language Modelscited by 19
  12. 2024On Information-Theoretic Measures of Predictive Uncertaintycited by 16
  13. 2023DUDES: Deep Uncertainty Distillation using Ensembles for Semantic Segmentationcited by 15
  14. 2023Uncertainty-based Fairness Measurescited by 15
  15. 2024Interpreting Predictive Probabilities: Model Confidence or Human Label Variation?cited by 15
  16. 2024(Implicit) Ensembles of Ensembles: Epistemic Uncertainty Collapse in Large Modelscited by 15
  17. 2023How Reliable is Your Regression Model's Uncertainty Under Real-World Distribution Shifts?cited by 14
  18. 2023A Data-Driven Measure of Relative Uncertainty for Misclassification Detectioncited by 14
  19. 2021Hyperdimensional Feature Fusion for Out-Of-Distribution Detectioncited by 13
  20. 2024Credal Wrapper of Model Averaging for Uncertainty Estimation in Classificationcited by 13
  21. 2025Cross-Modal and Uncertainty-Aware Agglomeration for Open-Vocabulary 3D Scene Understandingcited by 13
  22. 2023Density-Softmax: Efficient Test-time Model for Uncertainty Estimation and Robustness under Distribution Shiftscited by 12
  23. 2022Soft Augmentation for Image Classificationcited by 11
  24. 2023Beyond Distribution Shift: Spurious Features Through the Lens of Training Dynamicscited by 11
  25. 2023U-CE: Uncertainty-aware Cross-Entropy for Semantic Segmentationcited by 11
  26. 2025Uncertainty Quantification with the Empirical Neural Tangent Kernelcited by 11
  27. 2025Fast and Robust: Task Sampling with Posterior and Diversity Synergies for Adaptive Decision-Makers in Randomized Environmentscited by 11
  28. 2023Uncertainty Estimation on Sequential Labeling via Uncertainty Transmissioncited by 10
  29. 2025A Unified Evaluation Framework for Epistemic Predictionscited by 10
  30. 2025Are We Done with Object-Centric Learning?cited by 10
  31. 2025Uncertainty Estimation by Flexible Evidential Deep Learningcited by 10
  32. 2024Through the Lens of Split Vote: Exploring Disagreement, Difficulty and Calibration in Legal Case Outcome Classificationcited by 9
  33. 2024Uncertainty Quantification with Deep Ensembles for 6D Object Pose Estimationcited by 9
  34. 2024A Structured Review of Literature on Uncertainty in Machine Learning & Deep Learningcited by 9
  35. 2025Probabilistic Embeddings for Frozen Vision-Language Models: Uncertainty Quantification with Gaussian Process Latent Variable Modelscited by 9
  36. 2024SURE: SUrvey REcipes for building reliable and robust deep networkscited by 8
  37. 2025Epistemic Artificial Intelligence is Essential for Machine Learning Models to Truly 'Know When They Do Not Know'cited by 8
  38. 2022Raising the Bar on the Evaluation of Out-of-Distribution Detectioncited by 7

50 most cited of 200 All 200 (JSON)