GradPCA: Leveraging NTK Alignment for Reliable Out-of-Distribution Detection
Not on the map. Why?
Abstract (arXiv)
We introduce GradPCA, an Out-of-Distribution (OOD) detection method that exploits the low-rank structure of neural network gradients induced by Neural Tangent Kernel (NTK) alignment. GradPCA applies Principal Component Analysis (PCA) to gradient class-means, achieving more consistent performance than existing methods across standard image classification benchmarks. We provide a theoretical perspective on spectral OOD detection in neural networks to support GradPCA, highlighting feature-space properties that enable effective detection and naturally emerge from NTK alignment. Our analysis further reveals that feature quality -- particularly the use of pretrained versus non-pretrained representations -- plays a crucial role in determining which detectors will succeed. Extensive experiments validate the strong performance of GradPCA, and our theoretical framework offers guidance for designing more principled spectral OOD detectors.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 53 of its 79 references that are in the map. First posted on arXiv 21 May 2025.
Builds on
53 of 79 references.
References · 53
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2014Explaining and Harnessing Adversarial Examplescited by 7,364
- 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
- 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
- 2013Describing Textures in the Wildcited by 2,125
- 2016Adversarial examples in the physical worldcited by 1,769
- 2016Concrete Problems in AI Safetycited by 1,642
- 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,515
- 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,485
- 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,355
- 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,328
- 2020Energy-based Out-of-distribution Detectioncited by 1,299
38 more
- 2021Generalized Out-of-Distribution Detection: A Surveycited by 802
- 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 776
- 2019Big Transfer (BiT): General Visual Representation Learningcited by 769
- 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 719
- 2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 600
- 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 546
- 2021ReAct: Out-of-distribution Detection With Rectified Activationscited by 486
- 2019Likelihood Ratios for Out-of-Distribution Detectioncited by 453
- 2021On the Importance of Gradients for Detecting Distributional Shifts in the Wildcited by 339
- 2022VOS: Learning What You Don't Know by Virtual Outlier Synthesiscited by 300
- 2021SSD: A Unified Framework for Self-Supervised Outlier Detectioncited by 264
- 2021Knowledge distillation: A good teacher is patient and consistentcited by 259
- 2021MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Spacecited by 233
- 2020Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernelcited by 187
- 2023OpenOOD v1.5: Enhanced Benchmark for Out-of-Distribution Detectioncited by 179
- 2020Spectral Bias and Task-Model Alignment Explain Generalization in Kernel Regression and Infinitely Wide Neural Networkscited by 166
- 2021DICE: Leveraging Sparsification for Out-of-Distribution Detectioncited by 166
- 2015TurkerGaze: Crowdsourcing Saliency with Webcam based Eye Trackingcited by 165
- 2021A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challengescited by 154
- 2021Neural Networks as Kernel Learners: The Silent Alignment Effectcited by 106
- 2020Gradients as a Measure of Uncertainty in Neural Networkscited by 55
- 2020Backpropagated Gradient Representations for Anomaly Detectioncited by 54
- 2019The Local Elasticity of Neural Networkscited by 37
- 2023Detecting Out-of-Distribution Through the Lens of Neural Collapsecited by 34
- 2021No True State-of-the-Art? OOD Detection Methods are Inconsistent across Datasetscited by 31
- 2022Neural Tangent Kernel Beyond the Infinite-Width Limit: Effects of Depth and Initializationcited by 30
- 2021A Theory of Neural Tangent Kernel Alignment and Its Influence on Trainingcited by 25
- 2024Kernel PCA for Out-of-Distribution Detectioncited by 24
- 2023GradOrth: A Simple yet Efficient Out-of-Distribution Detection with Orthogonal Projection of Gradientscited by 21
- 2023Neural Collapse in the Intermediate Hidden Layers of Classification Neural Networkscited by 21
- 2020Label-Aware Neural Tangent Kernel: Toward Better Generalization and Local Elasticitycited by 18
- 2021Feature Learning and Signal Propagation in Deep Neural Networkscited by 18
- 2023Neural (Tangent Kernel) Collapsecited by 18
- 2022Gradient-Based Adversarial and Out-of-Distribution Detectioncited by 14
- 2024Gradient-Regularized Out-of-Distribution Detectioncited by 13
- 2023GAIA: Delving into Gradient-based Attribution Abnormality for Out-of-distribution Detectioncited by 12
- 2024Pursuing Feature Separation based on Neural Collapse for Out-of-Distribution Detectioncited by 12
- 2021Gradient-based Novelty Detection Boosted by Self-supervised Binary Classificationcited by 10
50 most cited of 53 All 53 (JSON)
Built on by
Citing papers per year: 2026: 2.
2 papers.