# GradOrth: A Simple yet Efficient Out-of-Distribution Detection with Orthogonal Projection of Gradients

arXiv: https://arxiv.org/abs/2308.00310 · HTML: https://mlsanity.com/paper/2308.00310/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 65 references that are in the map.
- First posted on arXiv 1 Aug 2023.

## Builds on (most cited first, 30 of 42)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Energy-based Out-of-distribution Detection](https://mlsanity.com/paper/2010.03759/) (2020)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases](https://mlsanity.com/paper/1705.02315/) (2017)
- [Generalized Out-of-Distribution Detection: A Survey](https://mlsanity.com/paper/2110.11334/) (2021)
- [Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images](https://mlsanity.com/paper/1412.1897/) (2014)
- [Out-of-Distribution Detection with Deep Nearest Neighbors](https://mlsanity.com/paper/2204.06507/) (2022)
- [ReAct: Out-of-distribution Detection With Rectified Activations](https://mlsanity.com/paper/2111.12797/) (2021)
- [Do Deep Generative Models Know What They Don't Know?](https://mlsanity.com/paper/1810.09136/) (2018)
- [Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Data](https://mlsanity.com/paper/2002.11297/) (2020)
- [On the Importance of Gradients for Detecting Distributional Shifts in the Wild](https://mlsanity.com/paper/2110.00218/) (2021)
- [MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Space](https://mlsanity.com/paper/2105.01879/) (2021)
- [Extremely Simple Activation Shaping for Out-of-Distribution Detection](https://mlsanity.com/paper/2209.09858/) (2022)
- [DICE: Leveraging Sparsification for Out-of-Distribution Detection](https://mlsanity.com/paper/2111.09805/) (2021)
- [TurkerGaze: Crowdsourcing Saliency with Webcam based Eye Tracking](https://mlsanity.com/paper/1504.06755/) (2015)
- [Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts?](https://mlsanity.com/paper/2006.14911/) (2020)
- [ATOM: Robustifying Out-of-distribution Detection Using Outlier Mining](https://mlsanity.com/paper/2006.15207/) (2020)
- [Convolutional neural networks with low-rank regularization](https://mlsanity.com/paper/1511.06067/) (2015)

## Built on by (most cited first, 21 of 21)

- [Out-of-Distribution Detection: A Task-Oriented Survey of Recent Advances](https://mlsanity.com/paper/2409.11884/) (2024)
- [CADRef: Robust Out-of-Distribution Detection via Class-Aware Decoupled Relative Feature Leveraging](https://mlsanity.com/paper/2503.00325/) (2025)
- [Taming False Positives in Out-of-Distribution Detection with Human Feedback](https://mlsanity.com/paper/2404.16954/) (2024)
- [AHA: Human-Assisted Out-of-Distribution Generalization and Detection](https://mlsanity.com/paper/2410.08000/) (2024)
- [AdaSCALE: Adaptive Scaling for OOD Detection](https://mlsanity.com/paper/2503.08023/) (2025)
- [Out-of-Distribution Learning with Human Feedback](https://mlsanity.com/paper/2408.07772/) (2024)
- [Adversarially Robust Out-of-Distribution Detection Using Lyapunov-Stabilized Embeddings](https://mlsanity.com/paper/2410.10744/) (2024)
- [GOOD: Training-Free Guided Diffusion Sampling for Out-of-Distribution Detection](https://mlsanity.com/paper/2510.17131/) (2025)
- [Your Classifier Can Be Secretly a Likelihood-Based OOD Detector](https://mlsanity.com/paper/2408.04851/) (2024)
- [Approximations to the Fisher Information Metric of Deep Generative Models for Out-Of-Distribution Detection](https://mlsanity.com/paper/2403.01485/) (2024)
- [GradPCA: Leveraging NTK Alignment for Reliable Out-of-Distribution Detection](https://mlsanity.com/paper/2505.16017/) (2025)
- [Activation Subspaces for Out-of-Distribution Detection](https://mlsanity.com/paper/2508.21695/) (2025)
- [LoD: Loss-difference OOD Detection by Intentionally Label-Noisifying Unlabeled Wild Data](https://mlsanity.com/paper/2505.12952/) (2025)
- [Kernel PCA for Out-of-Distribution Detection: Non-Linear Kernel Selection and Approximation](https://mlsanity.com/paper/2505.15284/) (2025)
- [Generalization vs. Memorization in Autoregressive Deep Learning: Or, Examining Temporal Decay of Gradient Coherence](https://mlsanity.com/paper/2509.00024/) (2025)
- [GEPC: Group-Equivariant Posterior Consistency for Out-of-Distribution Detection in Diffusion Models](https://mlsanity.com/paper/2602.00191/) (2026)
- [UniAlign: A Model-Agnostic Framework for Robust Network Traffic Classification under Distribution Shifts](https://mlsanity.com/paper/2605.17575/) (2026)
- [Catalyst: Out-of-Distribution Detection via Elastic Scaling](https://mlsanity.com/paper/2602.02409/) (2026)
- [The Invisible Gorilla Effect in Out-of-distribution Detection](https://mlsanity.com/paper/2602.20068/) (2026)
- [Tracking the Discriminative Axis: Dual Prototypes for Test-Time OOD Detection Under Covariate Shift](https://mlsanity.com/paper/2603.15213/) (2026)
- [Modality-Aware Out-of-Distribution Detection for Multi-Modal Action Recognition](https://mlsanity.com/paper/2606.24404/) (2026)

## Abstract (arXiv)

Detecting out-of-distribution (OOD) data is crucial for ensuring the safe deployment of machine learning models in real-world applications. However, existing OOD detection approaches primarily rely on the feature maps or the full gradient space information to derive OOD scores neglecting the role of most important parameters of the pre-trained network over in-distribution (ID) data. In this study, we propose a novel approach called GradOrth to facilitate OOD detection based on one intriguing observation that the important features to identify OOD data lie in the lower-rank subspace of in-distribution (ID) data. In particular, we identify OOD data by computing the norm of gradient projection on the subspaces considered important for the in-distribution data. A large orthogonal projection value (i.e. a small projection value) indicates the sample as OOD as it captures a weak correlation of the ID data. This simple yet effective method exhibits outstanding performance, showcasing a notable reduction in the average false positive rate at a 95% true positive rate (FPR95) of up to 8% when compared to the current state-of-the-art methods.