# Practical Evaluation of Out-of-Distribution Detection Methods for Image Classification

arXiv: https://arxiv.org/abs/2101.02447 · HTML: https://mlsanity.com/paper/2101.02447/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 41 references that are in the map.
- First posted on arXiv 7 Jan 2021.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Confidence Regularized Self-Training](https://mlsanity.com/paper/1908.09822/) (2019)
- [Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Data](https://mlsanity.com/paper/2002.11297/) (2020)
- [Rethinking ImageNet Pre-training](https://mlsanity.com/paper/1811.08883/) (2018)
- [Using Pre-Training Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1901.09960/) (2019)
- [Out-of-Distribution Detection Using an Ensemble of Self Supervised Leave-out Classifiers](https://mlsanity.com/paper/1809.03576/) (2018)
- [Unsupervised Out-of-Distribution Detection by Maximum Classifier Discrepancy](https://mlsanity.com/paper/1908.04951/) (2019)
- [Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes](https://mlsanity.com/paper/1707.09465/) (2017)
- [Unsupervised Domain Adaptation in Semantic Segmentation: a Review](https://mlsanity.com/paper/2005.10876/) (2020)
- [A Public Image Database for Benchmark of Plant Seedling Classification Algorithms](https://mlsanity.com/paper/1711.05458/) (2017)
- [Hyperparameter-Free Out-of-Distribution Detection Using Softmax of Scaled Cosine Similarity](https://mlsanity.com/paper/1905.10628/) (2019)

## Built on by (most cited first, 10 of 10)

- [Mixture Outlier Exposure: Towards Out-of-Distribution Detection in Fine-grained Environments](https://mlsanity.com/paper/2106.03917/) (2021)
- [Out-of-Distribution Detection for Medical Applications: Guidelines for Practical Evaluation](https://mlsanity.com/paper/2109.14885/) (2021)
- [Discretization-Induced Dirichlet Posterior for Robust Uncertainty Quantification on Regression](https://mlsanity.com/paper/2308.09065/) (2023)
- [Finding Competence Regions in Domain Generalization](https://mlsanity.com/paper/2303.09989/) (2023)
- [Natural Attribute-based Shift Detection](https://mlsanity.com/paper/2110.09276/) (2021)
- [ICONIC-444: A 3.1-Million-Image Dataset for OOD Detection Research](https://mlsanity.com/paper/2601.10802/) (2026)
- [BOOST: Out-of-Distribution-Informed Adaptive Sampling for Bias Mitigation in Stylistic Convolutional Neural Networks](https://mlsanity.com/paper/2507.07134/) (2025)
- [Benchmarking Out-of-Distribution Detection for Plankton Recognition: A Systematic Evaluation of Advanced Methods in Marine Ecological Monitoring](https://mlsanity.com/paper/2510.17179/) (2025)
- [Robust Uncertainty Estimation under Distribution Shift via Difference Reconstruction](https://mlsanity.com/paper/2601.19341/) (2026)
- [CUPID: A Plug-in Framework for Joint Aleatoric and Epistemic Uncertainty Estimation with a Single Model](https://mlsanity.com/paper/2603.10745/) (2026)

## Abstract (arXiv)

We reconsider the evaluation of OOD detection methods for image recognition. Although many studies have been conducted so far to build better OOD detection methods, most of them follow Hendrycks and Gimpel's work for the method of experimental evaluation. While the unified evaluation method is necessary for a fair comparison, there is a question of if its choice of tasks and datasets reflect real-world applications and if the evaluation results can generalize to other OOD detection application scenarios. In this paper, we experimentally evaluate the performance of representative OOD detection methods for three scenarios, i.e., irrelevant input detection, novel class detection, and domain shift detection, on various datasets and classification tasks. The results show that differences in scenarios and datasets alter the relative performance among the methods. Our results can also be used as a guide for practitioners for the selection of OOD detection methods.