mlsanityData to 7 Oct 2026

Bias-Reduced Uncertainty Estimation for Deep Neural Classifiers

arXiv 1805.08206 ↗First posted 21 May 2018cs.LG stat.MLcited by 106

Not on the map. Why?

Abstract (arXiv)

We consider the problem of uncertainty estimation in the context of (non-Bayesian) deep neural classification. In this context, all known methods are based on extracting uncertainty signals from a trained network optimized to solve the classification problem at hand. We demonstrate that such techniques tend to introduce biased estimates for instances whose predictions are supposed to be highly confident. We argue that this deficiency is an artifact of the dynamics of training with SGD-like optimizers, and it has some properties similar to overfitting. Based on this observation, we develop an uncertainty estimation algorithm that selectively estimates the uncertainty of highly confident points, using earlier snapshots of the trained model, before their estimates are jittered (and way before they are ready for actual classification). We present extensive experiments indicating that the proposed algorithm provides uncertainty estimates that are consistently better than all known methods.

Key facts. Cited by 106 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 12; 2023: 16; 2024: 23; 2025: 14; 2026: 31.

106 papers.

Sort:

By area:

  1. 2024Entropy is not Enough for Test-Time Adaptation: From the Perspective of Disentangled Factorscited by 106
  2. 2023Provable Dynamic Fusion for Low-Quality Multimodal Datacited by 81
  3. 2021Differentiable Learning Under Triagecited by 74
  4. 2022Conditional-Flow NeRF: Accurate 3D Modelling with Reliable Uncertainty Quantificationcited by 71
  5. 2022Confidence Score for Source-Free Unsupervised Domain Adaptationcited by 58
  6. 2021DOCTOR: A Simple Method for Detecting Misclassification Errorscited by 52
  7. 2022A Call to Reflect on Evaluation Practices for Failure Detection in Image Classificationcited by 51
  8. 2023Rethinking Confidence Calibration for Failure Predictioncited by 43
  9. 2023What Can We Learn From The Selective Prediction And Uncertainty Estimation Performance Of 523 Imagenet Classifierscited by 35
  10. 2022UMIX: Improving Importance Weighting for Subpopulation Shift via Uncertainty-Aware Mixupcited by 34
  11. 2024Smaller Language Models are capable of selecting Instruction-Tuning Training Data for Larger Language Modelscited by 32
  12. 2024Overcoming Common Flaws in the Evaluation of Selective Classification Systemscited by 31
38 more
  1. 2022Augmenting Softmax Information for Selective Classification with Out-of-Distribution Datacited by 27
  2. 2024ValUES: A Framework for Systematic Validation of Uncertainty Estimation in Semantic Segmentationcited by 25
  3. 2023Calibrating Uncertainty for Semi-Supervised Crowd Countingcited by 22
  4. 2022Selective Prediction via Training Dynamicscited by 20
  5. 2021Efficient Out-of-Distribution Detection Using Latent Space of β-VAE for Cyber-Physical Systemscited by 19
  6. 2021Disrupting Deep Uncertainty Estimation Without Harming Accuracycited by 19
  7. 2021A Unified Benchmark for the Unknown Detection Capability of Deep Neural Networkscited by 19
  8. 2022Respecting Transfer Gap in Knowledge Distillationcited by 18
  9. 2023The Devil is in the Wrongly-classified Samples: Towards Unified Open-set Recognitioncited by 16
  10. 2021Reinforcement Learning Under Algorithmic Triagecited by 15
  11. 2023Window-Based Early-Exit Cascades for Uncertainty Estimation: When Deep Ensembles are More Efficient than Single Modelscited by 15
  12. 2023How to Fix a Broken Confidence Estimator: Evaluating Post-hoc Methods for Selective Classification with Deep Neural Networkscited by 15
  13. 2023Average of Pruning: Improving Performance and Stability of Out-of-Distribution Detectioncited by 14
  14. 2024A Novel Characterization of the Population Area Under the Risk Coverage Curve (AURC) and Rates of Finite Sample Estimatorscited by 13
  15. 2024Selective Classification Under Distribution Shiftscited by 12
  16. 2022LOPS: Learning Order Inspired Pseudo-Label Selection for Weakly Supervised Text Classificationcited by 11
  17. 2024Towards Understanding Why Label Smoothing Degrades Selective Classification and How to Fix Itcited by 11
  18. 2023Temporal Knowledge Sharing enable Spiking Neural Network Learning from Past and Futurecited by 10
  19. 2021Dense Uncertainty Estimationcited by 9
  20. 2023Unified Classification and Rejection: A One-versus-All Frameworkcited by 9
  21. 2024Revisiting Confidence Estimation: Towards Reliable Failure Predictioncited by 9
  22. 2023Confidence Estimation Using Unlabeled Datacited by 8
  23. 2024SURE: SUrvey REcipes for building reliable and robust deep networkscited by 8
  24. 2022Toward Student-Oriented Teacher Network Training For Knowledge Distillationcited by 7
  25. 2024Tilt your Head: Activating the Hidden Spatial-Invariance of Classifierscited by 7
  26. 2024Hierarchical Selective Classificationcited by 7
  27. 2024Rate-In: Information-Driven Adaptive Dropout Rates for Improved Inference-Time Uncertainty Estimationcited by 7
  28. 2026Entropy Alone is Insufficient for Safe Selective Prediction in LLMscited by 7
  29. 2023Survey on Leveraging Uncertainty Estimation Towards Trustworthy Deep Neural Networks: The Case of Reject Option and Post-training Processingcited by 6
  30. 2024EPSD: Early Pruning with Self-Distillation for Efficient Model Compressioncited by 6
  31. 2024Typicalness-Aware Learning for Failure Detectioncited by 6
  32. 2025Spatially-Aware Evaluation of Segmentation Uncertaintycited by 6
  33. 2024Selective Learning: Towards Robust Calibration with Dynamic Regularizationcited by 5
  34. 2026The Cost of Reasoning: Chain-of-Thought Induces Overconfidence in Vision-Language Modelscited by 5
  35. 2023Empirical Optimal Risk to Quantify Model Trustworthiness for Failure Detectioncited by 4
  36. 2024Self-Attention through Kernel-Eigen Pair Sparse Variational Gaussian Processescited by 4
  37. 2025Suitability Filter: A Statistical Framework for Classifier Evaluation in Real-World Deployment Settingscited by 4
  38. 2025Decoding Uncertainty: The Impact of Decoding Strategies for Uncertainty Estimation in Large Language Modelscited by 3

50 most cited of 106 All 106 (JSON)