# Feature Space Singularity for Out-of-Distribution Detection

arXiv: https://arxiv.org/abs/2011.14654 · HTML: https://mlsanity.com/paper/2011.14654/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Nov 2020.

## Built on by (most cited first, 30 of 49)

- [Generalized Out-of-Distribution Detection: A Survey](https://mlsanity.com/paper/2110.11334/) (2021)
- [Deep Deterministic Uncertainty: A Simple Baseline](https://mlsanity.com/paper/2102.11582/) (2021)
- [A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation](https://mlsanity.com/paper/2305.11391/) (2023)
- [LINe: Out-of-Distribution Detection by Leveraging Important Neurons](https://mlsanity.com/paper/2303.13995/) (2023)
- [SAFE: Sensitivity-Aware Features for Out-of-Distribution Object Detection](https://mlsanity.com/paper/2208.13930/) (2022)
- [The Familiarity Hypothesis: Explaining the Behavior of Deep Open Set Methods](https://mlsanity.com/paper/2203.02486/) (2022)
- [Watermarking for Out-of-distribution Detection](https://mlsanity.com/paper/2210.15198/) (2022)
- [Neuron Activation Coverage: Rethinking Out-of-distribution Detection and Generalization](https://mlsanity.com/paper/2306.02879/) (2023)
- [FiLM-Ensemble: Probabilistic Deep Learning via Feature-wise Linear Modulation](https://mlsanity.com/paper/2206.00050/) (2022)
- [Out-of-Distribution Detection with Semantic Mismatch under Masking](https://mlsanity.com/paper/2208.00446/) (2022)
- [Rainproof: An Umbrella To Shield Text Generators From Out-Of-Distribution Data](https://mlsanity.com/paper/2212.09171/) (2022)
- [BadActs: A Universal Backdoor Defense in the Activation Space](https://mlsanity.com/paper/2405.11227/) (2024)
- [Expose Backdoors on the Way: A Feature-Based Efficient Defense against Textual Backdoor Attacks](https://mlsanity.com/paper/2210.07907/) (2022)
- [Unsupervised Layer-wise Score Aggregation for Textual OOD Detection](https://mlsanity.com/paper/2302.09852/) (2023)
- [A Practical Guide to Sample-based Statistical Distances for Evaluating Generative Models in Science](https://mlsanity.com/paper/2403.12636/) (2024)
- [Hyperdimensional Feature Fusion for Out-Of-Distribution Detection](https://mlsanity.com/paper/2112.05341/) (2021)
- [Conjugated Semantic Pool Improves OOD Detection with Pre-trained Vision-Language Models](https://mlsanity.com/paper/2410.08611/) (2024)
- [Fine-Tuning Deteriorates General Textual Out-of-Distribution Detection by Distorting Task-Agnostic Features](https://mlsanity.com/paper/2301.12715/) (2023)
- [3DOS: Towards 3D Open Set Learning -- Benchmarking and Understanding Semantic Novelty Detection on Point Clouds](https://mlsanity.com/paper/2207.11554/) (2022)
- [Extensible Machine Learning for Encrypted Network Traffic Application Labeling via Uncertainty Quantification](https://mlsanity.com/paper/2205.05628/) (2022)
- [MCROOD: Multi-Class Radar Out-Of-Distribution Detection](https://mlsanity.com/paper/2303.06232/) (2023)
- [Harnessing Unrecognizable Faces for Improving Face Recognition](https://mlsanity.com/paper/2106.04112/) (2021)
- [A Continual Learning Framework for Adaptive Defect Classification and Inspection](https://mlsanity.com/paper/2203.08796/) (2022)
- [Advancing Deep Active Learning & Data Subset Selection: Unifying Principles with Information-Theory Intuitions](https://mlsanity.com/paper/2401.04305/) (2024)
- [CVAD: A generic medical anomaly detector based on Cascade VAE](https://mlsanity.com/paper/2110.15811/) (2021)
- [Holistic Sentence Embeddings for Better Out-of-Distribution Detection](https://mlsanity.com/paper/2210.07485/) (2022)
- [Reconstruction-based Out-of-Distribution Detection for Short-Range FMCW Radar](https://mlsanity.com/paper/2302.14192/) (2023)
- [Diffusion-based Layer-wise Semantic Reconstruction for Unsupervised Out-of-Distribution Detection](https://mlsanity.com/paper/2411.10701/) (2024)
- [HOOD: Real-Time Human Presence and Out-of-Distribution Detection Using FMCW Radar](https://mlsanity.com/paper/2308.02396/) (2023)
- [HAROOD: Human Activity Classification and Out-of-Distribution Detection with Short-Range FMCW Radar](https://mlsanity.com/paper/2312.08894/) (2023)

## Abstract (arXiv)

Out-of-Distribution (OoD) detection is important for building safe artificial intelligence systems. However, current OoD detection methods still cannot meet the performance requirements for practical deployment. In this paper, we propose a simple yet effective algorithm based on a novel observation: in a trained neural network, OoD samples with bounded norms well concentrate in the feature space. We call the center of OoD features the Feature Space Singularity (FSS), and denote the distance of a sample feature to FSS as FSSD. Then, OoD samples can be identified by taking a threshold on the FSSD. Our analysis of the phenomenon reveals why our algorithm works. We demonstrate that our algorithm achieves state-of-the-art performance on various OoD detection benchmarks. Besides, FSSD also enjoys robustness to slight corruption in test data and can be further enhanced by ensembling. These make FSSD a promising algorithm to be employed in real world. We release our code at urlhttps://github.com/megvii-research/FSSDOoDDetection.