# Gradients as a Measure of Uncertainty in Neural Networks

arXiv: https://arxiv.org/abs/2008.08030 · HTML: https://mlsanity.com/paper/2008.08030/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Aug 2020.

## Built on by (most cited first, 30 of 55)

- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [On the Importance of Gradients for Detecting Distributional Shifts in the Wild](https://mlsanity.com/paper/2110.00218/) (2021)
- [A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directions](https://mlsanity.com/paper/2412.05563/) (2024)
- [LUQ: Long-text Uncertainty Quantification for LLMs](https://mlsanity.com/paper/2403.20279/) (2024)
- [Single-model uncertainty quantification in neural network potentials does not consistently outperform model ensembles](https://mlsanity.com/paper/2305.01754/) (2023)
- [Gradient-based Uncertainty for Monocular Depth Estimation](https://mlsanity.com/paper/2208.02005/) (2022)
- [Out-of-Distribution Detection: A Task-Oriented Survey of Recent Advances](https://mlsanity.com/paper/2409.11884/) (2024)
- [A Comprehensive Survey on Evidential Deep Learning and Its Applications](https://mlsanity.com/paper/2409.04720/) (2024)
- [GradOrth: A Simple yet Efficient Out-of-Distribution Detection with Orthogonal Projection of Gradients](https://mlsanity.com/paper/2308.00310/) (2023)
- [How Useful are Gradients for OOD Detection Really?](https://mlsanity.com/paper/2205.10439/) (2022)
- [Introspective Learning : A Two-Stage Approach for Inference in Neural Networks](https://mlsanity.com/paper/2209.08425/) (2022)
- [Predictive Dynamic Fusion](https://mlsanity.com/paper/2406.04802/) (2024)
- [Fine-grained TLS services classification with reject option](https://mlsanity.com/paper/2202.11984/) (2022)
- [Gradient-Based Adversarial and Out-of-Distribution Detection](https://mlsanity.com/paper/2206.08255/) (2022)
- [Ctrl-U: Robust Conditional Image Generation via Uncertainty-aware Reward Modeling](https://mlsanity.com/paper/2410.11236/) (2024)
- [Extracting Causal Visual Features for Limited label Classification](https://mlsanity.com/paper/2103.12322/) (2021)
- [GAIA: Delving into Gradient-based Attribution Abnormality for Out-of-distribution Detection](https://mlsanity.com/paper/2311.09620/) (2023)
- [Contrastive Reasoning in Neural Networks](https://mlsanity.com/paper/2103.12329/) (2021)
- [Explanatory Paradigms in Neural Networks](https://mlsanity.com/paper/2202.11838/) (2022)
- [Gradient-based Novelty Detection Boosted by Self-supervised Binary Classification](https://mlsanity.com/paper/2112.09815/) (2021)
- [Gaussian Switch Sampling: A Second Order Approach to Active Learning](https://mlsanity.com/paper/2302.12018/) (2023)
- [Revisiting Deep Ensemble for Out-of-Distribution Detection: A Loss Landscape Perspective](https://mlsanity.com/paper/2310.14227/) (2023)
- [Epistemic Uncertainty Quantification For Pre-trained Neural Network](https://mlsanity.com/paper/2404.10124/) (2024)
- [A Structured Review of Literature on Uncertainty in Machine Learning & Deep Learning](https://mlsanity.com/paper/2406.00332/) (2024)
- [Highlighting Object Category Immunity for the Generalization of Human-Object Interaction Detection](https://mlsanity.com/paper/2202.09492/) (2022)
- [WDiscOOD: Out-of-Distribution Detection via Whitened Linear Discriminant Analysis](https://mlsanity.com/paper/2303.07543/) (2023)
- [Probing the Purview of Neural Networks via Gradient Analysis](https://mlsanity.com/paper/2304.02834/) (2023)
- [Harnessing Uncertainty-aware Bounding Boxes for Unsupervised 3D Object Detection](https://mlsanity.com/paper/2408.00619/) (2024)
- [Forgetful Active Learning with Switch Events: Efficient Sampling for Out-of-Distribution Data](https://mlsanity.com/paper/2301.05106/) (2023)
- [Open-Set Recognition with Gradient-Based Representations](https://mlsanity.com/paper/2206.08229/) (2022)

## Abstract (arXiv)

Despite tremendous success of modern neural networks, they are known to be overconfident even when the model encounters inputs with unfamiliar conditions. Detecting such inputs is vital to preventing models from making naive predictions that may jeopardize real-world applications of neural networks. In this paper, we address the challenging problem of devising a simple yet effective measure of uncertainty in deep neural networks. Specifically, we propose to utilize backpropagated gradients to quantify the uncertainty of trained models. Gradients depict the required amount of change for a model to properly represent given inputs, thus providing a valuable insight into how familiar and certain the model is regarding the inputs. We demonstrate the effectiveness of gradients as a measure of model uncertainty in applications of detecting unfamiliar inputs, including out-of-distribution and corrupted samples. We show that our gradient-based method outperforms state-of-the-art methods by up to 4.8% of AUROC score in out-of-distribution detection and 35.7% in corrupted input detection.