# Learning for Single-Shot Confidence Calibration in Deep Neural Networks through Stochastic Inferences

arXiv: https://arxiv.org/abs/1809.10877 · HTML: https://mlsanity.com/paper/1809.10877/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Sep 2018.

## Built on by (most cited first, 25 of 25)

- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety](https://mlsanity.com/paper/2104.14235/) (2021)
- [No Cost Likelihood Manipulation at Test Time for Making Better Mistakes in Deep Networks](https://mlsanity.com/paper/2104.00795/) (2021)
- [Uncertainty-Calibrated Test-Time Model Adaptation without Forgetting](https://mlsanity.com/paper/2403.11491/) (2024)
- [Logic Traps in Evaluating Attribution Scores](https://mlsanity.com/paper/2109.05463/) (2021)
- [MBCT: Tree-Based Feature-Aware Binning for Individual Uncertainty Calibration](https://mlsanity.com/paper/2202.04348/) (2022)
- [Self-Distribution Distillation: Efficient Uncertainty Estimation](https://mlsanity.com/paper/2203.08295/) (2022)
- [A Holistic Assessment of the Reliability of Machine Learning Systems](https://mlsanity.com/paper/2307.10586/) (2023)
- [From Black-box to White-box: Examining Confidence Calibration under different Conditions](https://mlsanity.com/paper/2101.02971/) (2021)
- [Confidence-based Estimators for Predictive Performance in Model Monitoring](https://mlsanity.com/paper/2407.08649/) (2024)
- [Bayesian Confidence Calibration for Epistemic Uncertainty Modelling](https://mlsanity.com/paper/2109.10092/) (2021)
- [A Hybrid of Generative and Discriminative Models Based on the Gaussian-coupled Softmax Layer](https://mlsanity.com/paper/2305.05912/) (2023)
- [Towards Calibrated Deep Clustering Network](https://mlsanity.com/paper/2403.02998/) (2024)
- [SLOVA: Uncertainty Estimation Using Single Label One-Vs-All Classifier](https://mlsanity.com/paper/2206.13923/) (2022)
- [Generalized Uncertainty of Deep Neural Networks: Taxonomy and Applications](https://mlsanity.com/paper/2302.01440/) (2023)
- [Saccadic Vision for Fine-Grained Visual Classification](https://mlsanity.com/paper/2509.15688/) (2025)
- [Hidden Heterogeneity: When to Choose Similarity-Based Calibration](https://mlsanity.com/paper/2202.01840/) (2022)
- [Propagating Variational Model Uncertainty for Bioacoustic Call Label Smoothing](https://mlsanity.com/paper/2210.10526/) (2022)
- [Perception and Semantic Aware Regularization for Sequential Confidence Calibration](https://mlsanity.com/paper/2305.19498/) (2023)
- [Confidence Calibration with Bounded Error Using Transformations](https://mlsanity.com/paper/2102.12680/) (2021)
- [Loss Estimators Improve Model Generalization](https://mlsanity.com/paper/2103.03788/) (2021)
- [Rethinking Soft Label in Label Distribution Learning Perspective](https://mlsanity.com/paper/2301.13444/) (2023)
- [Pseudo-label Learning with Calibrated Confidence Using an Energy-based Model](https://mlsanity.com/paper/2404.09585/) (2024)
- [Low-Regret and Low-Complexity Learning for Hierarchical Inference](https://mlsanity.com/paper/2508.08985/) (2025)
- [Efficient Analytic Uncertainty Quantification for Multi-Modal Regression](https://mlsanity.com/paper/2606.25188/) (2026)

## Abstract (arXiv)

We propose a generic framework to calibrate accuracy and confidence of a prediction in deep neural networks through stochastic inferences. We interpret stochastic regularization using a Bayesian model, and analyze the relation between predictive uncertainty of networks and variance of the prediction scores obtained by stochastic inferences for a single example. Our empirical study shows that the accuracy and the score of a prediction are highly correlated with the variance of multiple stochastic inferences given by stochastic depth or dropout. Motivated by this observation, we design a novel variance-weighted confidence-integrated loss function that is composed of two cross-entropy loss terms with respect to ground-truth and uniform distribution, which are balanced by variance of stochastic prediction scores. The proposed loss function enables us to learn deep neural networks that predict confidence calibrated scores using a single inference. Our algorithm presents outstanding confidence calibration performance and improves classification accuracy when combined with two popular stochastic regularization techniques---stochastic depth and dropout---in multiple models and datasets; it alleviates overconfidence issue in deep neural networks significantly by training networks to achieve prediction accuracy proportional to confidence of prediction.