# Uncertainty Estimation Using a Single Deep Deterministic Neural Network

arXiv: https://arxiv.org/abs/2003.02037 · HTML: https://mlsanity.com/paper/2003.02037/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Mar 2020.

## Built on by (most cited first, 30 of 37)

- [SegmentMeIfYouCan: A Benchmark for Anomaly Segmentation](https://mlsanity.com/paper/2104.14812/) (2021)
- [DEUP: Direct Epistemic Uncertainty Prediction](https://mlsanity.com/paper/2102.08501/) (2021)
- [The Promises and Pitfalls of Deep Kernel Learning](https://mlsanity.com/paper/2102.12108/) (2021)
- [Understanding Softmax Confidence and Uncertainty](https://mlsanity.com/paper/2106.04972/) (2021)
- [Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertainties](https://mlsanity.com/paper/2103.08951/) (2021)
- [Post-hoc Uncertainty Learning using a Dirichlet Meta-Model](https://mlsanity.com/paper/2212.07359/) (2022)
- [Corner Cases for Visual Perception in Automated Driving: Some Guidance on Detection Approaches](https://mlsanity.com/paper/2102.05897/) (2021)
- [Uncertainty-based Fairness Measures](https://mlsanity.com/paper/2312.11299/) (2023)
- [Interpreting Predictive Probabilities: Model Confidence or Human Label Variation?](https://mlsanity.com/paper/2402.16102/) (2024)
- [Towards OOD Detection in Graph Classification from Uncertainty Estimation Perspective](https://mlsanity.com/paper/2206.10691/) (2022)
- [The Peril of Popular Deep Learning Uncertainty Estimation Methods](https://mlsanity.com/paper/2112.05000/) (2021)
- [Uncertainty-Aware Credit Card Fraud Detection Using Deep Learning](https://mlsanity.com/paper/2107.13508/) (2021)
- [Provable Uncertainty Decomposition via Higher-Order Calibration](https://mlsanity.com/paper/2412.18808/) (2024)
- [Semi-supervised Deep Learning for Image Classification with Distribution Mismatch: A Survey](https://mlsanity.com/paper/2203.00190/) (2022)
- [Function Space Diversity for Uncertainty Prediction via Repulsive Last-Layer Ensembles](https://mlsanity.com/paper/2412.15758/) (2024)
- [Advancing Deep Active Learning & Data Subset Selection: Unifying Principles with Information-Theory Intuitions](https://mlsanity.com/paper/2401.04305/) (2024)
- [OCCUQ: Exploring Efficient Uncertainty Quantification for 3D Occupancy Prediction](https://mlsanity.com/paper/2503.10605/) (2025)
- [Towards Robust Active Feature Acquisition](https://mlsanity.com/paper/2107.04163/) (2021)
- [Confidence Aware Neural Networks for Skin Cancer Detection](https://mlsanity.com/paper/2107.09118/) (2021)
- [Distributional Gaussian Process Layers for Outlier Detection in Image Segmentation](https://mlsanity.com/paper/2104.13756/) (2021)
- [Invariant Representation Driven Neural Classifier for Anti-QCD Jet Tagging](https://mlsanity.com/paper/2201.07199/) (2022)
- [Improving Generalization via Uncertainty Driven Perturbations](https://mlsanity.com/paper/2202.05737/) (2022)
- [Dealing with Distribution Mismatch in Semi-supervised Deep Learning for Covid-19 Detection Using Chest X-ray Images: A Novel Approach Using Feature Densities](https://mlsanity.com/paper/2109.00889/) (2021)
- [Distinction Maximization Loss: Efficiently Improving Out-of-Distribution Detection and Uncertainty Estimation by Replacing the Loss and Calibrating](https://mlsanity.com/paper/2205.05874/) (2022)
- [Generalized Uncertainty of Deep Neural Networks: Taxonomy and Applications](https://mlsanity.com/paper/2302.01440/) (2023)
- [Implicit regularisation in stochastic gradient descent: from single-objective to two-player games](https://mlsanity.com/paper/2307.05789/) (2023)
- [Automatic quality control framework for more reliable integration of machine learning-based image segmentation into medical workflows](https://mlsanity.com/paper/2112.03277/) (2021)
- [Distributional Gaussian Processes Layers for Out-of-Distribution Detection](https://mlsanity.com/paper/2206.13346/) (2022)
- [Detecting and Learning Out-of-Distribution Data in the Open world: Algorithm and Theory](https://mlsanity.com/paper/2310.06221/) (2023)
- [Masked Gamma-SSL: Learning Uncertainty Estimation via Masked Image Modeling](https://mlsanity.com/paper/2402.17622/) (2024)

## Abstract (arXiv)

We propose a method for training a deterministic deep model that can find and reject out of distribution data points at test time with a single forward pass. Our approach, deterministic uncertainty quantification (DUQ), builds upon ideas of RBF networks. We scale training in these with a novel loss function and centroid updating scheme and match the accuracy of softmax models. By enforcing detectability of changes in the input using a gradient penalty, we are able to reliably detect out of distribution data. Our uncertainty quantification scales well to large datasets, and using a single model, we improve upon or match Deep Ensembles in out of distribution detection on notable difficult dataset pairs such as FashionMNIST vs. MNIST, and CIFAR-10 vs. SVHN.