mlsanityData to 7 Oct 2026

How Useful are Gradients for OOD Detection Really?

arXiv 2205.10439 ↗First posted 20 May 2022cs.LGcited by 20

Not on the map. Why?

Abstract (arXiv)

One critical challenge in deploying highly performant machine learning models in real-life applications is out of distribution (OOD) detection. Given a predictive model which is accurate on in distribution (ID) data, an OOD detection system will further equip the model with the option to defer prediction when the input is novel and the model has little confidence in prediction. There has been some recent interest in utilizing the gradient information in pre-trained models for OOD detection. While these methods have shown competitive performance, there are misconceptions about the true mechanism underlying them, which conflate their performance with the necessity of gradients. In this work, we provide an in-depth analysis and comparison of gradient based methods and elucidate the key components that warrant their OOD detection performance. We further propose a general, non-gradient based method of OOD detection which improves over previous baselines in both performance and computational efficiency.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 22 of its 42 references that are in the map. First posted on arXiv 20 May 2022.

Builds on

22 of 42 references.

References · 22

  1. 2017On Calibration of Modern Neural Networkscited by 4,860
  2. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  3. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  4. 2013Describing Textures in the Wildcited by 2,110
  5. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  6. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  7. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  8. 2020Energy-based Out-of-distribution Detectioncited by 1,296
  9. 2018Deep Anomaly Detection with Outlier Exposurecited by 939
  10. 2021Generalized Out-of-Distribution Detection: A Surveycited by 796
  11. 2019Big Transfer (BiT): General Visual Representation Learningcited by 766
  12. 2019Likelihood Ratios for Out-of-Distribution Detectioncited by 451
10 more
  1. 2017Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samplescited by 389
  2. 2021On the Importance of Gradients for Detecting Distributional Shifts in the Wildcited by 338
  3. 2018Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problemcited by 323
  4. 2021MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Spacecited by 233
  5. 2019Input complexity and out-of-distribution detection with likelihood-based generative modelscited by 186
  6. 2021A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challengescited by 154
  7. 2019Unsupervised Out-of-Distribution Detection by Maximum Classifier Discrepancycited by 112
  8. 2020Estimating Example Difficulty Using Variance of Gradientscited by 96
  9. 2020Gradients as a Measure of Uncertainty in Neural Networkscited by 55
  10. 2019Out-of-distribution Detection in Classifiers via Generationcited by 49

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 5; 2025: 5; 2026: 2.

20 papers.

Sort:

By area:

  1. 2024A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directionscited by 115
  2. 2023Learning to Augment Distributions for Out-of-Distribution Detectioncited by 38
  3. 2023Out-of-distribution Detection Learning with Unreliable Out-of-distribution Sourcescited by 31
  4. 2023GradOrth: A Simple yet Efficient Out-of-Distribution Detection with Orthogonal Projection of Gradientscited by 21
  5. 2023GAIA: Delving into Gradient-based Attribution Abnormality for Out-of-distribution Detectioncited by 12
  6. 2024Pursuing Feature Separation based on Neural Collapse for Out-of-Distribution Detectioncited by 12
  7. 2025Controlling Neural Collapse Enhances Out-of-Distribution Detection and Transfer Learningcited by 11
  8. 2023Meta OOD Learning for Continuously Adaptive OOD Detectioncited by 10
  9. 2024Epistemic Uncertainty Quantification For Pre-trained Neural Networkcited by 10
  10. 2023Low-Dimensional Gradient Helps Out-of-Distribution Detectioncited by 6
  11. 2025Digital Twin-based Out-of-Distribution Detection in Autonomous Vesselscited by 3
  12. 2023T2FNorm: Extremely Simple Scaled Train-time Feature Normalization for OOD Detectioncited by 2
8 more
  1. 2024Approximations to the Fisher Information Metric of Deep Generative Models for Out-Of-Distribution Detectioncited by 2
  2. 2025STOOD-X methodology: using statistical nonparametric test for OOD Detection Large-Scale datasets enhanced with explainabilitycited by 2
  3. 2025An Empirical Analysis of VLM-based OOD Detection: Mechanisms, Advantages, and Sensitivitycited by 1
  4. 2023GROOD: GRadient-Aware Out-of-Distribution Detectioncited by 0
  5. 2024Concept Matching with Agent for Out-of-Distribution Detectioncited by 0
  6. 2025Probabilistic Runtime Verification, Evaluation and Risk Assessment of Visual Deep Learning Systemscited by 0
  7. 2026Gradients with Respect to Semantics Preserving Embeddings Tell the Uncertainty of Large Language Modelscited by 0
  8. 2026Reliable Neural Collapse Approximation for Open-World Test-Time Adaptationcited by 0