mlsanityData to 7 Oct 2026

RankFeat: Rank-1 Feature Removal for Out-of-distribution Detection

arXiv 2209.08590 ↗First posted 18 Sep 2022cs.LG cs.CVcited by 52

Not on the map. Why?

Abstract (arXiv)

The task of out-of-distribution (OOD) detection is crucial for deploying machine learning models in real-world settings. In this paper, we observe that the singular value distributions of the in-distribution (ID) and OOD features are quite different: the OOD feature matrix tends to have a larger dominant singular value than the ID feature, and the class predictions of OOD samples are largely determined by it. This observation motivates us to propose textttRankFeat, a simple yet effective textttpost hoc approach for OOD detection by removing the rank-1 matrix composed of the largest singular value and the associated singular vectors from the high-level feature (i.e., X- s1u1v1T). textttRankFeat achieves the state-of-the-art performance and reduces the average false positive rate (FPR95) by 17.90% compared with the previous best method. Extensive ablation studies and comprehensive theoretical analyses are presented to support the empirical results.

Key facts. Cited by 52 papers in the map (data through 7 Oct 2026). Builds on 55 of its 78 references that are in the map. First posted on arXiv 18 Sep 2022.

Builds on

55 of 78 references.

References · 55

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  5. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  6. 2013Describing Textures in the Wildcited by 2,110
  7. 2016Density estimation using Real NVPcited by 2,029
  8. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  9. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  10. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
  11. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  12. 2020Energy-based Out-of-distribution Detectioncited by 1,296
38 more
  1. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  2. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
  3. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,054
  4. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 897
  5. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  6. 2021Generalized Out-of-Distribution Detection: A Surveycited by 796
  7. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
  8. 2019Big Transfer (BiT): General Visual Representation Learningcited by 766
  9. 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 722
  10. 2015Towards Open Set Deep Networkscited by 533
  11. 2021RepVGG: Making VGG-style ConvNets Great Againcited by 520
  12. 2021ReAct: Out-of-distribution Detection With Rectified Activationscited by 486
  13. 2019Scaling Out-of-Distribution Detection for Real-World Settingscited by 465
  14. 2019Likelihood Ratios for Out-of-Distribution Detectioncited by 451
  15. 2018Do Deep Generative Models Know What They Don't Know?cited by 443
  16. 2020CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instancescited by 441
  17. 2020Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Datacited by 408
  18. 2021Adaptive Conformal Inference Under Distribution Shiftcited by 360
  19. 2021Open-Set Recognition: a Good Closed-Set Classifier is All You Need?cited by 351
  20. 2021On the Importance of Gradients for Detecting Distributional Shifts in the Wildcited by 338
  21. 2022VOS: Learning What You Don't Know by Virtual Outlier Synthesiscited by 300
  22. 2021Exploring the Limits of Out-of-Distribution Detectioncited by 283
  23. 2018Deep Anomaly Detection Using Geometric Transformationscited by 269
  24. 2021MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Spacecited by 233
  25. 2019Learning to Discover Novel Visual Categories via Deep Transfer Clusteringcited by 223
  26. 2020Why Normalizing Flows Fail to Detect Out-of-Distribution Datacited by 220
  27. 2019Input complexity and out-of-distribution detection with likelihood-based generative modelscited by 186
  28. 2021A Fine-Grained Analysis on Distribution Shiftcited by 171
  29. 2018Latent Space Autoregression for Novelty Detectioncited by 149
  30. 2020Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encodercited by 145
  31. 2022Leveraging Unlabeled Data to Predict Out-of-Distribution Performancecited by 141
  32. 2021Neighborhood Contrastive Learning for Novel Class Discoverycited by 119
  33. 2021Can multi-label classification networks know what they don't know?cited by 100
  34. 2021Towards a Theoretical Framework of Out-of-Distribution Generalizationcited by 94
  35. 2020Understanding Anomaly Detection with Deep Invertible Networks through Hierarchies of Distributions and Featurescited by 64
  36. 2021A Winning Hand: Compressing Deep Networks Can Improve Out-Of-Distribution Robustnesscited by 62
  37. 2016Stacked Generative Adversarial Networkscited by 44
  38. 2020Further Analysis of Outlier Detection with Deep Generative Modelscited by 31

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 15; 2024: 16; 2025: 17; 2026: 3.

52 papers.

Sort:

  1. 2023OpenOOD v1.5: Enhanced Benchmark for Out-of-Distribution Detectioncited by 178
  2. 2023In or Out? Fixing ImageNet Out-of-Distribution Detection Evaluationcited by 132
  3. 2023Nearest Neighbor Guidance for Out-of-Distribution Detectioncited by 64
  4. 2023NECO: NEural Collapse Based Out-of-distribution detectioncited by 41
  5. 2024Outlier detection by ensembling uncertainty with negative objectnesscited by 34
  6. 2022Block Selection Method for Using Feature Norm in Out-of-distribution Detectioncited by 30
  7. 2023Neuron Activation Coverage: Rethinking Out-of-distribution Detection and Generalizationcited by 28
  8. 2023DIVERSIFY: A General Framework for Time Series Out-of-distribution Detection and Generalizationcited by 24
  9. 2024Kernel PCA for Out-of-Distribution Detectioncited by 24
  10. 2024Learning Structured Representations with Hyperbolic Embeddingscited by 16
  11. 2023VRA: Variational Rectified Activation for Out-of-distribution Detectioncited by 15
  12. 2023On the use of Mahalanobis distance for out-of-distribution detection with neural networks for medical imagingcited by 15
38 more
  1. 2024Towards Optimal Feature-Shaping Methods for Out-of-Distribution Detectioncited by 13
  2. 2024A noisy elephant in the room: Is your out-of-distribution detector robust to label noise?cited by 13
  3. 2025OpenMIBOOD: Open Medical Imaging Benchmarks for Out-Of-Distribution Detectioncited by 13
  4. 2023GAIA: Delving into Gradient-based Attribution Abnormality for Out-of-distribution Detectioncited by 12
  5. 2023Revisiting Deep Ensemble for Out-of-Distribution Detection: A Loss Landscape Perspectivecited by 10
  6. 2025EigenTrack: Spectral Activation Feature Tracking for Hallucination and Out-of-Distribution Detection in LLMs and VLMscited by 10
  7. 2024A Unified Approach Towards Active Learning and Out-of-Distribution Detectioncited by 7
  8. 2024CLIP-driven Outliers Synthesis for few-shot OOD detectioncited by 6
  9. 2024SCOD: From Heuristics to Theorycited by 3
  10. 2024Improving Transferable Targeted Adversarial Attack via Normalized Logit Calibration and Truncated Feature Mixingcited by 3
  11. 2024Skeleton-OOD: An End-to-End Skeleton-Based Model for Robust Out-of-Distribution Human Action Detectioncited by 3
  12. 2024Rethinking the Evaluation of Out-of-Distribution Detection: A Sorites Paradoxcited by 3
  13. 2025Enhancing Out-of-Distribution Detection with Extended Logit Normalizationcited by 3
  14. 2023A Functional Data Perspective and Baseline On Multi-Layer Out-of-Distribution Detectioncited by 2
  15. 2023Reject option models comprising out-of-distribution detectioncited by 2
  16. 2025OpenEarthSensing: Large-Scale Fine-Grained Benchmark for Open-World Remote Sensingcited by 2
  17. 2025Open-Insect: Benchmarking Open-Set Recognition of Novel Species in Biodiversity Monitoringcited by 2
  18. 2025SpectralGap: Graph-Level Out-of-Distribution Detection via Laplacian Eigenvalue Gapscited by 2
  19. 2025TULiP: Test-time Uncertainty Estimation via Linearization and Weight Perturbationcited by 2
  20. 2025Activation Subspaces for Out-of-Distribution Detectioncited by 2
  21. 2023RankFeat&RankWeight: Rank-1 Feature/Weight Removal for Out-of-distribution Detectioncited by 1
  22. 2023ExCeL : Combined Extreme and Collective Logit Information for Enhancing Out-of-Distribution Detectioncited by 1
  23. 2024Combine and Conquer: A Meta-Analysis on Data Shift and Out-of-Distribution Detectioncited by 1
  24. 2024Representation Norm Amplification for Out-of-Distribution Detection in Long-Tail Learningcited by 1
  25. 2024Prototype-based Optimal Transport for Out-of-Distribution Detectioncited by 1
  26. 2025Advancing Out-of-Distribution Detection via Local Neuroplasticitycited by 1
  27. 2025Kernel PCA for Out-of-Distribution Detection: Non-Linear Kernel Selection and Approximationcited by 1
  28. 2025BootOOD: Self-Supervised Out-of-Distribution Detection via Synthetic Sample Exposure under Neural Collapsecited by 1
  29. 2026ICONIC-444: A 3.1-Million-Image Dataset for OOD Detection Researchcited by 1
  30. 2023GROOD: GRadient-Aware Out-of-Distribution Detectioncited by 0
  31. 2024Dimensionality-induced information loss of outliers in deep neural networkscited by 0
  32. 2024Semantic or Covariate? A Study on the Intractable Case of Out-of-Distribution Detectioncited by 0
  33. 2025Optimizing Latent Dimension Allocation in Hierarchical VAEs: Balancing Attenuation and Information Retention for OOD Detectioncited by 0
  34. 2025Feature Bank Enhancement for Distance-based Out-of-Distribution Detectioncited by 0
  35. 2025Pseudo-label Induced Subspace Representation Learning for Robust Out-of-Distribution Detectioncited by 0
  36. 2025Long-Tailed Out-of-Distribution Detection with Refined Separate Class Learningcited by 0
  37. 2025Benchmarking Out-of-Distribution Detection for Plankton Recognition: A Systematic Evaluation of Advanced Methods in Marine Ecological Monitoringcited by 0
  38. 2025TSRE: Channel-Aware Typical Set Refinement for Out-of-Distribution Detectioncited by 0

50 most cited of 52 All 52 (JSON)