# A Systematic Analysis of Out-of-Distribution Detection Under Representation and Training Paradigm Shifts

arXiv: https://arxiv.org/abs/2511.11934 · HTML: https://mlsanity.com/paper/2511.11934/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 54 references that are in the map.
- First posted on arXiv 14 Nov 2025.

## Builds on (most cited first, 29 of 29)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [Energy-based Out-of-distribution Detection](https://mlsanity.com/paper/2010.03759/) (2020)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Prevalence of Neural Collapse during the terminal phase of deep learning training](https://mlsanity.com/paper/2008.08186/) (2020)
- [Scaling Out-of-Distribution Detection for Real-World Settings](https://mlsanity.com/paper/1911.11132/) (2019)
- [ViM: Out-Of-Distribution with Virtual-logit Matching](https://mlsanity.com/paper/2203.10807/) (2022)
- [On the Importance of Gradients for Detecting Distributional Shifts in the Wild](https://mlsanity.com/paper/2110.00218/) (2021)
- [The Intrinsic Dimension of Images and Its Impact on Learning](https://mlsanity.com/paper/2104.08894/) (2021)
- [OpenOOD: Benchmarking Generalized Out-of-Distribution Detection](https://mlsanity.com/paper/2210.07242/) (2022)
- [Learning Confidence for Out-of-Distribution Detection in Neural Networks](https://mlsanity.com/paper/1802.04865/) (2018)
- [Addressing Failure Prediction by Learning Model Confidence](https://mlsanity.com/paper/1910.04851/) (2019)
- [OpenOOD v1.5: Enhanced Benchmark for Out-of-Distribution Detection](https://mlsanity.com/paper/2306.09301/) (2023)
- [Bias-Reduced Uncertainty Estimation for Deep Neural Classifiers](https://mlsanity.com/paper/1805.08206/) (2018)
- [Deep Gamblers: Learning to Abstain with Portfolio Theory](https://mlsanity.com/paper/1907.00208/) (2019)
- [Nearest Neighbor Guidance for Out-of-Distribution Detection](https://mlsanity.com/paper/2309.14888/) (2023)
- [DOCTOR: A Simple Method for Detecting Misclassification Errors](https://mlsanity.com/paper/2106.02395/) (2021)
- [A Call to Reflect on Evaluation Practices for Failure Detection in Image Classification](https://mlsanity.com/paper/2211.15259/) (2022)
- [Fast Decision Boundary based Out-of-Distribution Detector](https://mlsanity.com/paper/2312.11536/) (2023)
- [NECO: NEural Collapse Based Out-of-distribution detection](https://mlsanity.com/paper/2310.06823/) (2023)
- [Overcoming Common Flaws in the Evaluation of Selective Classification Systems](https://mlsanity.com/paper/2407.01032/) (2024)
- [Confidence Estimation via Auxiliary Models](https://mlsanity.com/paper/2012.06508/) (2020)
- [Single Layer Predictive Normalized Maximum Likelihood for Out-of-Distribution Detection](https://mlsanity.com/paper/2110.09246/) (2021)
- [OpenMIBOOD: Open Medical Imaging Benchmarks for Out-Of-Distribution Detection](https://mlsanity.com/paper/2503.16247/) (2025)
- [Understanding Silent Failures in Medical Image Classification](https://mlsanity.com/paper/2307.14729/) (2023)
- [A Cosine Similarity-based Method for Out-of-Distribution Detection](https://mlsanity.com/paper/2306.14920/) (2023)
- [Kernel PCA for Out-of-Distribution Detection: Non-Linear Kernel Selection and Approximation](https://mlsanity.com/paper/2505.15284/) (2025)

## Abstract (arXiv)

We present a systematic benchmark of out-of-distribution (OOD) detection CSFs through a representation-centric lens. Our study spans CNN and ViT backbones, multiple training paradigms, four image-classification source datasets (CIFAR-10, CIFAR-100, SuperCIFAR-100, and TinyImageNet), and OOD datasets grouped into near, mid, and far regimes using CLIP-derived semantic distances. To compare CSFs across these settings, we employ a multiple-comparison-controlled rank pipeline that identifies top cliques of statistically indistinguishable winners under threshold-free ranking metrics (AURC and AUGRC). The main empirical finding is that the competitive detector family depends more on the learned representation than on score design alone. For both CNNs and ViTs, simple probabilistic scores dominate misclassification detection. On CNNs, margin-based scores are strongest in near-OOD regimes, while geometry-aware scores such as NNGuide, fDBD, and CTM become more competitive as shift severity increases. On fine-tuned ViTs, the top cliques are led mainly by reconstruction- and residual-based scores. To interpret these ranking shifts, we analyze the last-layer representation using Neural Collapse (NC) metrics. The resulting picture is consistent across architectures: prototype- and boundary-aware scores become stronger when the representation is more collapsed and better aligned with classifier weights, whereas weaker-collapse regimes favor gradient- and manifold-based scores. Building on these insights, we propose two contributions: a simple PCA-based projection-filtering procedure that improves detector performance, and an approach that uses NC measurements computed from a trained classifier to predict its competitive out-of-distribution detector shortlist, without requiring any additional OOD data.