mlsanityData to 14 Oct 2026

No True State-of-the-Art? OOD Detection Methods are Inconsistent across Datasets

arXiv 2109.05554 ↗First posted 12 Sep 2021cs.LGcited by 31

Not on the map. Why?

Abstract (arXiv)

Out-of-distribution detection is an important component of reliable ML systems. Prior literature has proposed various methods (e.g., MSP (Hendrycks & Gimpel, 2017), ODIN (Liang et al., 2018), Mahalanobis (Lee et al., 2018)), claiming they are state-of-the-art by showing they outperform previous methods on a selected set of in-distribution (ID) and out-of-distribution (OOD) datasets. In this work, we show that none of these methods are inherently better at OOD detection than others on a standardized set of 16 (ID, OOD) pairs. We give possible explanations for these inconsistencies with simple toy datasets where whether one method outperforms another depends on the structure of the ID and OOD datasets in question. Finally, we show that a method outperforming another on a certain (ID, OOD) pair may not do so in a low-data regime. In the low-data regime, we propose a distance-based method, Pairwise OOD detection (POD), which is based on Siamese networks and improves over Mahalanobis by sidestepping the expensive covariance estimation step. Our results suggest that the OOD detection problem may be too broad, and we should consider more specific structures for leverage.

Key facts. Cited by 31 papers in the map (data through 14 Oct 2026). Builds on 33 of its 56 references that are in the map. First posted on arXiv 12 Sep 2021.

Builds on

33 of 56 references.

References · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  5. 2013Intriguing properties of neural networkscited by 5,155
  6. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  7. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  8. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  9. 2016Wide Residual Networkscited by 3,070
  10. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  11. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  12. 2016Concrete Problems in AI Safetycited by 1,642
21 more
  1. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,485
  2. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,355
  3. 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,331
  4. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,328
  5. 2020Energy-based Out-of-distribution Detectioncited by 1,299
  6. 2018Deep Anomaly Detection with Outlier Exposurecited by 942
  7. 2016Universal adversarial perturbationscited by 784
  8. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 776
  9. 2020A Unifying Review of Deep and Shallow Anomaly Detectioncited by 425
  10. 2020Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Datacited by 408
  11. 2020Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awarenesscited by 389
  12. 2020Consistent Estimators for Learning to Defer to an Expertcited by 271
  13. 2019Revisiting Self-Supervised Visual Representation Learningcited by 251
  14. 2021MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Spacecited by 233
  15. 2021A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detectioncited by 207
  16. 2019Detecting semantic anomaliescited by 50
  17. 2020SelfAugment: Automatic Augmentation Policies for Self-Supervised Learningcited by 32
  18. 2020CheXpedition: Investigating Generalization Challenges for Translation of Chest X-Ray Algorithms to the Clinical Settingcited by 18
  19. 2018Prototypical Clustering Networks for Dermatological Disease Diagnosiscited by 6
  20. 2019Open Set Medical Diagnosiscited by 6
  21. 2021CheXseen: Unseen Disease Detection for Deep Learning Interpretation of Chest X-rayscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 4; 2023: 5; 2024: 5; 2025: 9; 2026: 7.

31 papers.

Sort:

By area:

  1. 2023OpenOOD v1.5: Enhanced Benchmark for Out-of-Distribution Detectioncited by 179
  2. 2021A Unified Benchmark for the Unknown Detection Capability of Deep Neural Networkscited by 19
  3. 2025Out-of-Distribution Detection Methods Answer the Wrong Questionscited by 14
  4. 2023WDiscOOD: Out-of-Distribution Detection via Whitened Linear Discriminant Analysiscited by 9
  5. 2023Conservative Prediction via Data-Driven Confidence Minimizationcited by 8
  6. 2025Bridging Streaming Continual Learning via In-Context Large Tabular Modelscited by 7
  7. 2022Out of Distribution Detection on ImageNet-Ocited by 5
  8. 2024COOD: Combined out-of-distribution detection using multiple measures for anomaly & novel class detection in large-scale hierarchical classificationcited by 5
  9. 2024WeiPer: OOD Detection using Weight Perturbations of Class Projectionscited by 4
  10. 2025Improving Out-of-Distribution Detection via Dynamic Covariance Calibrationcited by 4
  11. 2025BOOD: Boundary-based Out-Of-Distribution Data Generationcited by 4
  12. 2024Building Real-time Awareness of Out-of-distribution in Trajectory Prediction for Autonomous Vehiclescited by 3
19 more
  1. 2025A Recipe for Causal Graph Regression: Confounding Effects Revisitedcited by 3
  2. 2023Interpretable Out-Of-Distribution Detection Using Pattern Identificationcited by 2
  3. 2024Monitizer: Automating Design and Evaluation of Neural Network Monitorscited by 2
  4. 2025STOOD-X methodology: using statistical nonparametric test for OOD Detection Large-Scale datasets enhanced with explainabilitycited by 2
  5. 2025GradPCA: Leveraging NTK Alignment for Reliable Out-of-Distribution Detectioncited by 2
  6. 2026COMBOOD: A Semiparametric Approach for Detecting Out-of-distribution Data for Image Classificationcited by 2
  7. 2022XOOD: Extreme Value Based Out-Of-Distribution Detection For Image Classificationcited by 1
  8. 2022Contrastive Novelty-Augmented Learning: Anticipating Outliers with Large Language Modelscited by 1
  9. 2023A Comprehensive Assessment Benchmark for Rigorously Evaluating Deep Learning Image Classifierscited by 1
  10. 2024Toward a Realistic Benchmark for Out-of-Distribution Detectioncited by 1
  11. 2026TASTE: Task-Aware Out-of-Distribution Detection via Stein Operatorscited by 1
  12. 2026Scaling Pretrained Representations Enables Label-Free Out-of-Distribution Detection Without Fine-Tuningcited by 1
  13. 2022Simple Calibration via Geodesic Kernelscited by 0
  14. 2025Out of Distribution Detection for Efficient Continual Learning in Quality Prediction for Arc Weldingcited by 0
  15. 2025Benchmarking Out-of-Distribution Detection for Plankton Recognition: A Systematic Evaluation of Advanced Methods in Marine Ecological Monitoringcited by 0
  16. 2026The Invisible Gorilla Effect in Out-of-distribution Detectioncited by 0
  17. 2026Bounding Box Anomaly Scoring for simple and efficient Out-of-Distribution detectioncited by 0
  18. 2026Few-Shot Out of Domain Intent Detection with Covariance Corrected Mahalanobis Distancecited by 0
  19. 2026Dataset Identity, Not Novelty: The Source of an Inflated OOD Detection Gaincited by 0