# Revisiting One-vs-All Classifiers for Predictive Uncertainty and Out-of-Distribution Detection in Neural Networks

arXiv: https://arxiv.org/abs/2007.05134 · HTML: https://mlsanity.com/paper/2007.05134/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jul 2020.

## Built on by (most cited first, 30 of 31)

- [OVANet: One-vs-All Network for Universal Domain Adaptation](https://mlsanity.com/paper/2104.03344/) (2021)
- [A Stitch in Time Saves Nine: A Train-Time Regularizing Loss for Improved Neural Network Calibration](https://mlsanity.com/paper/2203.13834/) (2022)
- [A Simple Approach to Improve Single-Model Deep Uncertainty via Distance-Awareness](https://mlsanity.com/paper/2205.00403/) (2022)
- [Energy-Based Open-World Uncertainty Modeling for Confidence Calibration](https://mlsanity.com/paper/2107.12628/) (2021)
- [OpenMatch: Open-set Consistency Regularization for Semi-supervised Learning with Outliers](https://mlsanity.com/paper/2105.14148/) (2021)
- [Harnessing Out-Of-Distribution Examples via Augmenting Content and Style](https://mlsanity.com/paper/2207.03162/) (2022)
- [Expanding Low-Density Latent Regions for Open-Set Object Detection](https://mlsanity.com/paper/2203.14911/) (2022)
- [Towards Improving Calibration in Object Detection Under Domain Shift](https://mlsanity.com/paper/2209.07601/) (2022)
- [Robust Semi-supervised Learning by Wisely Leveraging Open-set Data](https://mlsanity.com/paper/2405.06979/) (2024)
- [Latent Discriminant deterministic Uncertainty](https://mlsanity.com/paper/2207.10130/) (2022)
- [UQGAN: A Unified Model for Uncertainty Quantification of Deep Classifiers trained via Conditional GANs](https://mlsanity.com/paper/2201.13279/) (2022)
- [Uncertainty-Induced Transferability Representation for Source-Free Unsupervised Domain Adaptation](https://mlsanity.com/paper/2208.13986/) (2022)
- [Unified Classification and Rejection: A One-versus-All Framework](https://mlsanity.com/paper/2311.13355/) (2023)
- [Training on Test Data with Bayesian Adaptation for Covariate Shift](https://mlsanity.com/paper/2109.12746/) (2021)
- [SURE: SUrvey REcipes for building reliable and robust deep networks](https://mlsanity.com/paper/2403.00543/) (2024)
- [One-vs-the-Rest Loss to Focus on Important Samples in Adversarial Training](https://mlsanity.com/paper/2207.10283/) (2022)
- [Improving CLIP Robustness with Knowledge Distillation and Self-Training](https://mlsanity.com/paper/2309.10361/) (2023)
- [Distributional Gaussian Process Layers for Outlier Detection in Image Segmentation](https://mlsanity.com/paper/2104.13756/) (2021)
- [Invariant Representation Driven Neural Classifier for Anti-QCD Jet Tagging](https://mlsanity.com/paper/2201.07199/) (2022)
- [UR2M: Uncertainty and Resource-Aware Event Detection on Microcontrollers](https://mlsanity.com/paper/2402.09264/) (2024)
- [Transitional Uncertainty with Layered Intermediate Predictions](https://mlsanity.com/paper/2405.17494/) (2024)
- [Deep Joint Distribution Optimal Transport for Universal Domain Adaptation on Time Series](https://mlsanity.com/paper/2503.11217/) (2025)
- [SLOVA: Uncertainty Estimation Using Single Label One-Vs-All Classifier](https://mlsanity.com/paper/2206.13923/) (2022)
- [Towards Adaptive Unknown Authentication for Universal Domain Adaptation by Classifier Paradox](https://mlsanity.com/paper/2207.04494/) (2022)
- [Handling Label Uncertainty for Camera Incremental Person Re-Identification](https://mlsanity.com/paper/2210.08710/) (2022)
- [Low-Resource Named Entity Recognition: Can One-vs-All AUC Maximization Help?](https://mlsanity.com/paper/2311.04918/) (2023)
- [DEUCE: Dual-diversity Enhancement and Uncertainty-awareness for Cold-start Active Learning](https://mlsanity.com/paper/2502.00305/) (2025)
- [Distributional Gaussian Processes Layers for Out-of-Distribution Detection](https://mlsanity.com/paper/2206.13346/) (2022)
- [Multiclass Alignment of Confidence and Certainty for Network Calibration](https://mlsanity.com/paper/2309.02636/) (2023)
- [Embracing Unknown Step by Step: Towards Reliable Sparse Training in Real World](https://mlsanity.com/paper/2403.20047/) (2024)

## Abstract (arXiv)

Accurate estimation of predictive uncertainty in modern neural networks is critical to achieve well calibrated predictions and detect out-of-distribution (OOD) inputs. The most promising approaches have been predominantly focused on improving model uncertainty (e.g. deep ensembles and Bayesian neural networks) and post-processing techniques for OOD detection (e.g. ODIN and Mahalanobis distance). However, there has been relatively little investigation into how the parametrization of the probabilities in discriminative classifiers affects the uncertainty estimates, and the dominant method, softmax cross-entropy, results in misleadingly high confidences on OOD data and under covariate shift. We investigate alternative ways of formulating probabilities using (1) a one-vs-all formulation to capture the notion of "none of the above", and (2) a distance-based logit representation to encode uncertainty as a function of distance to the training manifold. We show that one-vs-all formulations can improve calibration on image classification tasks, while matching the predictive performance of softmax without incurring any additional training or test-time complexity.