mlsanityData to 7 Oct 2026

On the Importance of Gradients for Detecting Distributional Shifts in the Wild

arXiv 2110.00218 ↗First posted 1 Oct 2021cs.LGcited by 338

Not on the map. Why?

Cited by 21 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Detecting out-of-distribution (OOD) data has become a critical component in ensuring the safe deployment of machine learning models in the real world. Existing OOD detection approaches primarily rely on the output or feature space for deriving OOD scores, while largely overlooking information from the gradient space. In this paper, we present GradNorm, a simple and effective approach for detecting OOD inputs by utilizing information extracted from the gradient space. GradNorm directly employs the vector norm of gradients, backpropagated from the KL divergence between the softmax output and a uniform probability distribution. Our key idea is that the magnitude of gradients is higher for in-distribution (ID) data than that for OOD data, making it informative for OOD detection. GradNorm demonstrates superior performance, reducing the average FPR95 by up to 16.33% compared to the previous best method.

Key facts. Cited by 338 papers in the map (data through 7 Oct 2026). Builds on 37 of its 57 references that are in the map. First posted on arXiv 1 Oct 2021.

Builds on

37 of 57 references.

References · 37

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2016Densely Connected Convolutional Networkscited by 6,691
  4. 2013Auto-Encoding Variational Bayescited by 3,980
  5. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  6. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  7. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  8. 2013Describing Textures in the Wildcited by 2,110
  9. 2016Density estimation using Real NVPcited by 2,029
  10. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  11. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  12. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
25 more
  1. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  2. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  3. 2020Energy-based Out-of-distribution Detectioncited by 1,296
  4. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  5. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  6. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
  7. 2019Big Transfer (BiT): General Visual Representation Learningcited by 766
  8. 2021Domain Generalization: A Surveycited by 760
  9. 2015Towards Open Set Deep Networkscited by 533
  10. 2019Likelihood Ratios for Out-of-Distribution Detectioncited by 451
  11. 2018Do Deep Generative Models Know What They Don't Know?cited by 443
  12. 2020Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Datacited by 408
  13. 2021MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Spacecited by 233
  14. 2020Why Normalizing Flows Fail to Detect Out-of-Distribution Datacited by 220
  15. 2019Input complexity and out-of-distribution detection with likelihood-based generative modelscited by 186
  16. 2020Contrastive Training for Improved Out-of-Distribution Detectioncited by 186
  17. 2020Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encodercited by 145
  18. 2020ATOM: Robustifying Out-of-distribution Detection Using Outlier Miningcited by 116
  19. 2021Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Taskscited by 100
  20. 2021Can multi-label classification networks know what they don't know?cited by 100
  21. 2021MOOD: Multi-level Out-of-distribution Detectioncited by 94
  22. 2020Understanding Anomaly Detection with Deep Invertible Networks through Hierarchies of Distributions and Featurescited by 64
  23. 2020Gradients as a Measure of Uncertainty in Neural Networkscited by 55
  24. 2016Stacked Generative Adversarial Networkscited by 44
  25. 2020Further Analysis of Outlier Detection with Deep Generative Modelscited by 31

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 43; 2023: 80; 2024: 97; 2025: 67; 2026: 45.

338 papers.

Sort:

By area:

  1. 2021Generalized Out-of-Distribution Detection: A Surveycited by 796
  2. 2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 596
  3. 2021ReAct: Out-of-distribution Detection With Rectified Activationscited by 486
  4. 2022ViM: Out-Of-Distribution with Virtual-logit Matchingcited by 404
  5. 2022VOS: Learning What You Don't Know by Virtual Outlier Synthesiscited by 300
  6. 2022OpenOOD: Benchmarking Generalized Out-of-Distribution Detectioncited by 267
  7. 2022Mitigating Neural Network Overconfidence with Logit Normalizationcited by 251
  8. 2022Delving into Out-of-Distribution Detection with Vision-Language Representationscited by 215
  9. 2023OpenOOD v1.5: Enhanced Benchmark for Out-of-Distribution Detectioncited by 178
  10. 2021DICE: Leveraging Sparsification for Out-of-Distribution Detectioncited by 166
  11. 2021A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challengescited by 154
  12. 2023Non-Parametric Outlier Synthesiscited by 129
38 more
  1. 2022How to Exploit Hyperspherical Embeddings for Out-of-Distribution Detection?cited by 128
  2. 2024A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directionscited by 115
  3. 2022Is Out-of-Distribution Detection Learnable?cited by 109
  4. 2023LoCoOp: Few-Shot Out-of-Distribution Detection via Prompt Learningcited by 107
  5. 2024ARGS: Alignment as Reward-Guided Searchcited by 99
  6. 2022Training OOD Detectors in their Natural Habitatscited by 86
  7. 2021Provable Guarantees for Understanding Out-of-distribution Detectioncited by 78
  8. 2024Process Reward Model with Q-Value Rankingscited by 78
  9. 2023Dream the Impossible: Outlier Imagination with Diffusion Modelscited by 77
  10. 2022Unknown-Aware Object Detection: Learning What You Don't Know from Videos in the Wildcited by 70
  11. 2024Negative Label Guided OOD Detection with Pretrained Vision-Language Modelscited by 66
  12. 2023Nearest Neighbor Guidance for Out-of-Distribution Detectioncited by 64
  13. 2023Unsupervised Out-of-Distribution Detection with Diffusion Inpaintingcited by 57
  14. 2023Feed Two Birds with One Scone: Exploiting Wild Data for Both Out-of-Distribution Generalization and Detectioncited by 57
  15. 2022RankFeat: Rank-1 Feature Removal for Out-of-distribution Detectioncited by 52
  16. 2022Boosting Out-of-distribution Detection with Typical Featurescited by 48
  17. 2023Fast Decision Boundary based Out-of-Distribution Detectorcited by 48
  18. 2023Rethinking Out-of-distribution (OOD) Detection: Masked Image Modeling is All You Needcited by 46
  19. 2024Learning with Mixture of Prototypes for Out-of-Distribution Detectioncited by 43
  20. 2024Learning Transferable Negative Prompts for Out-of-Distribution Detectioncited by 43
  21. 2023NECO: NEural Collapse Based Out-of-distribution detectioncited by 41
  22. 2023Diversified Outlier Exposure for Out-of-Distribution Detection via Informative Extrapolationcited by 38
  23. 2023Learning to Augment Distributions for Out-of-Distribution Detectioncited by 38
  24. 2022Towards Understanding and Boosting Adversarial Transferability from a Distribution Perspectivecited by 36
  25. 2023GL-MCM: Global and Local Maximum Concept Matching for Zero-Shot Out-of-Distribution Detectioncited by 36
  26. 2024How Does Unlabeled Data Provably Help Out-of-Distribution Detection?cited by 36
  27. 2024Outlier detection by ensembling uncertainty with negative objectnesscited by 34
  28. 2022OpenCon: Open-world Contrastive Learningcited by 33
  29. 2022SAFE: Sensitivity-Aware Features for Out-of-Distribution Object Detectioncited by 33
  30. 2023Detecting Out-of-Distribution Through the Lens of Neural Collapsecited by 33
  31. 2023Out-of-distribution Detection Learning with Unreliable Out-of-distribution Sourcescited by 31
  32. 2022Block Selection Method for Using Feature Norm in Out-of-distribution Detectioncited by 30
  33. 2023ImageNet-E: Benchmarking Neural Network Robustness via Attribute Editingcited by 30
  34. 2024Envisioning Outlier Exposure by Large Language Models for Out-of-Distribution Detectioncited by 30
  35. 2022Watermarking for Out-of-distribution Detectioncited by 28
  36. 2023Neuron Activation Coverage: Rethinking Out-of-distribution Detection and Generalizationcited by 28
  37. 2022Augmenting Softmax Information for Selective Classification with Out-of-Distribution Datacited by 27
  38. 2022Gradient-based Uncertainty for Monocular Depth Estimationcited by 26

50 most cited of 338 All 338 (JSON)