# Dropout Distillation for Efficiently Estimating Model Confidence

arXiv: https://arxiv.org/abs/1809.10562 · HTML: https://mlsanity.com/paper/1809.10562/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Sep 2018.

## Built on by (most cited first, 9 of 9)

- [From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligence](https://mlsanity.com/paper/2110.15245/) (2021)
- [Robust Models are less Over-Confident](https://mlsanity.com/paper/2210.05938/) (2022)
- [Self-Knowledge Distillation via Dropout](https://mlsanity.com/paper/2208.05642/) (2022)
- [Fool Me Once: Robust Selective Segmentation via Out-of-Distribution Detection with Contrastive Learning](https://mlsanity.com/paper/2103.00869/) (2021)
- [Learning Uncertainty For Safety-Oriented Semantic Segmentation In Autonomous Driving](https://mlsanity.com/paper/2105.13688/) (2021)
- [Efficient Multi-task Uncertainties for Joint Semantic Segmentation and Monocular Depth Estimation](https://mlsanity.com/paper/2402.10580/) (2024)
- [Generalized Uncertainty of Deep Neural Networks: Taxonomy and Applications](https://mlsanity.com/paper/2302.01440/) (2023)
- [FoMo X: Modular Explainability Signals for Outlier Detection Foundation Models](https://mlsanity.com/paper/2603.17570/) (2026)
- [SGD-Based Knowledge Distillation with Bayesian Teachers: Theory and Guidelines](https://mlsanity.com/paper/2601.01484/) (2026)

## Abstract (arXiv)

We propose an efficient way to output better calibrated uncertainty scores from neural networks. The Distilled Dropout Network (DDN) makes standard (non-Bayesian) neural networks more introspective by adding a new training loss which prevents them from being overconfident. Our method is more efficient than Bayesian neural networks or model ensembles which, despite providing more reliable uncertainty scores, are more cumbersome to train and slower to test. We evaluate DDN on the the task of image classification on the CIFAR-10 dataset and show that our calibration results are competitive even when compared to 100 Monte Carlo samples from a dropout network while they also increase the classification accuracy. We also propose better calibration within the state of the art Faster R-CNN object detection framework and show, using the COCO dataset, that DDN helps train better calibrated object detectors.