# Dissecting Out-of-Distribution Detection and Open-Set Recognition: A Critical Analysis of Methods and Benchmarks

arXiv: https://arxiv.org/abs/2408.16757 · HTML: https://mlsanity.com/paper/2408.16757/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 56 references that are in the map.
- First posted on arXiv 29 Aug 2024.

## Builds on (most cited first, 30 of 37)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [Natural Adversarial Examples](https://mlsanity.com/paper/1907.07174/) (2019)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Energy-based Out-of-distribution Detection](https://mlsanity.com/paper/2010.03759/) (2020)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Deep Anomaly Detection with Outlier Exposure](https://mlsanity.com/paper/1812.04606/) (2018)
- [Generalized Out-of-Distribution Detection: A Survey](https://mlsanity.com/paper/2110.11334/) (2021)
- [ImageNet-21K Pretraining for the Masses](https://mlsanity.com/paper/2104.10972/) (2021)
- [ReAct: Out-of-distribution Detection With Rectified Activations](https://mlsanity.com/paper/2111.12797/) (2021)
- [Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Data](https://mlsanity.com/paper/2002.11297/) (2020)
- [Open-Set Recognition: a Good Closed-Set Classifier is All You Need?](https://mlsanity.com/paper/2110.06207/) (2021)
- [On the Importance of Gradients for Detecting Distributional Shifts in the Wild](https://mlsanity.com/paper/2110.00218/) (2021)
- [OpenOOD: Benchmarking Generalized Out-of-Distribution Detection](https://mlsanity.com/paper/2210.07242/) (2022)
- [Extremely Simple Activation Shaping for Out-of-Distribution Detection](https://mlsanity.com/paper/2209.09858/) (2022)
- [A Fine-Grained Analysis on Distribution Shift](https://mlsanity.com/paper/2110.11328/) (2021)
- [Generative OpenMax for Multi-Class Open Set Classification](https://mlsanity.com/paper/1707.07418/) (2017)
- [Adversarial Reciprocal Points Learning for Open Set Recognition](https://mlsanity.com/paper/2103.00953/) (2021)
- [OpenGAN: Open-Set Recognition via Open Data Generation](https://mlsanity.com/paper/2104.02939/) (2021)
- [ATOM: Robustifying Out-of-distribution Detection Using Outlier Mining](https://mlsanity.com/paper/2006.15207/) (2020)
- [Learning Open Set Network with Discriminative Reciprocal Points](https://mlsanity.com/paper/2011.00178/) (2020)
- [POEM: Out-of-Distribution Detection with Posterior Sampling](https://mlsanity.com/paper/2206.13687/) (2022)
- [Semantically Coherent Out-of-Distribution Detection](https://mlsanity.com/paper/2108.11941/) (2021)
- [Plex: Towards Reliability using Pretrained Large Model Extensions](https://mlsanity.com/paper/2207.07411/) (2022)

## Built on by (most cited first, 13 of 13)

- [DebGCD: Debiased Learning with Distribution Guidance for Generalized Category Discovery](https://mlsanity.com/paper/2504.04804/) (2025)
- [When to retrain a machine learning model](https://mlsanity.com/paper/2505.14903/) (2025)
- [Leveraging Intermediate Representations for Better Out-of-Distribution Detection](https://mlsanity.com/paper/2502.12849/) (2025)
- [Looking Beyond the Known: Towards a Data Discovery Guided Open-World Object Detection](https://mlsanity.com/paper/2510.00303/) (2025)
- [OpenEarthSensing: Large-Scale Fine-Grained Benchmark for Open-World Remote Sensing](https://mlsanity.com/paper/2502.20668/) (2025)
- [SphOR: A Representation Learning Perspective on Open-set Recognition for Identifying Unknown Classes in Deep Learning Models](https://mlsanity.com/paper/2503.08049/) (2025)
- [v-CLR: View-Consistent Learning for Open-World Instance Segmentation](https://mlsanity.com/paper/2504.01383/) (2025)
- [Drainage: A Unifying Framework for Addressing Class Uncertainty](https://mlsanity.com/paper/2512.03182/) (2025)
- [One Model, Many Behaviors: Training-Induced Effects on Out-of-Distribution Detection](https://mlsanity.com/paper/2601.10836/) (2026)
- [Layout-Aware Representation Learning for Open-Set ID Fraud Discovery](https://mlsanity.com/paper/2605.05215/) (2026)
- [Benchmarking Out-of-Distribution Detection for Plankton Recognition: A Systematic Evaluation of Advanced Methods in Marine Ecological Monitoring](https://mlsanity.com/paper/2510.17179/) (2025)
- [GEODE: Angle-Adaptive OOD Detection with Universal Scorer Compatibility](https://mlsanity.com/paper/2605.01063/) (2026)
- [Dataset Identity, Not Novelty: The Source of an Inflated OOD Detection Gain](https://mlsanity.com/paper/2610.01096/) (2026)

## Abstract (arXiv)

Detecting test-time distribution shift has emerged as a key capability for safely deployed machine learning models, with the question being tackled under various guises in recent years. In this paper, we aim to provide a consolidated view of the two largest sub-fields within the community: out-of-distribution (OOD) detection and open-set recognition (OSR). In particular, we aim to provide rigorous empirical analysis of different methods across settings and provide actionable takeaways for practitioners and researchers. Concretely, we make the following contributions: (i) We perform rigorous cross-evaluation between state-of-the-art methods in the OOD detection and OSR settings and identify a strong correlation between the performances of methods for them; (ii) We propose a new, large-scale benchmark setting which we suggest better disentangles the problem tackled by OOD detection and OSR, re-evaluating state-of-the-art OOD detection and OSR methods in this setting; (iii) We surprisingly find that the best performing method on standard benchmarks (Outlier Exposure) struggles when tested at scale, while scoring rules which are sensitive to the deep feature magnitude consistently show promise; and (iv) We conduct empirical analysis to explain these phenomena and highlight directions for future research. Code: https://github.com/Visual-AI/Dissect-OOD-OSR