# Epistemic Uncertainty Quantification For Pre-trained Neural Network

arXiv: https://arxiv.org/abs/2404.10124 · HTML: https://mlsanity.com/paper/2404.10124/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 40 references that are in the map.
- First posted on arXiv 15 Apr 2024.

## Builds on (most cited first, 25 of 25)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [A Simple Baseline for Bayesian Uncertainty in Deep Learning](https://mlsanity.com/paper/1902.02476/) (2019)
- [Predictive Uncertainty Estimation via Prior Networks](https://mlsanity.com/paper/1802.10501/) (2018)
- [Laplace Redux -- Effortless Bayesian Deep Learning](https://mlsanity.com/paper/2106.14806/) (2021)
- [Stochastic Gradient Hamiltonian Monte Carlo](https://mlsanity.com/paper/1402.4102/) (2014)
- [On the Importance of Gradients for Detecting Distributional Shifts in the Wild](https://mlsanity.com/paper/2110.00218/) (2021)
- [Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks](https://mlsanity.com/paper/2002.10118/) (2020)
- [Cyclical Stochastic Gradient MCMC for Bayesian Deep Learning](https://mlsanity.com/paper/1902.03932/) (2019)
- [Ensemble Distribution Distillation](https://mlsanity.com/paper/1905.00076/) (2019)
- [Multiplicative Normalizing Flows for Variational Bayesian Neural Networks](https://mlsanity.com/paper/1703.01961/) (2017)
- [Bayesian Deep Learning via Subnetwork Inference](https://mlsanity.com/paper/2010.14689/) (2020)
- [Bayesian Uncertainty Estimation for Batch Normalized Deep Networks](https://mlsanity.com/paper/1802.06455/) (2018)
- [On the Practicality of Deterministic Epistemic Uncertainty](https://mlsanity.com/paper/2107.00649/) (2021)
- [Gradients as a Measure of Uncertainty in Neural Networks](https://mlsanity.com/paper/2008.08030/) (2020)
- [Sketching Curvature for Efficient Out-of-Distribution Detection for Deep Neural Networks](https://mlsanity.com/paper/2102.12567/) (2021)
- [Estimating Model Uncertainty of Neural Networks in Sparse Information Form](https://mlsanity.com/paper/2006.11631/) (2020)
- [Training-Free Uncertainty Estimation for Dense Regression: Sensitivity as a Surrogate](https://mlsanity.com/paper/1910.04858/) (2019)
- [Quantification of Uncertainty with Adversarial Models](https://mlsanity.com/paper/2307.03217/) (2023)
- [How Useful are Gradients for OOD Detection Really?](https://mlsanity.com/paper/2205.10439/) (2022)
- [Encoding the latent posterior of Bayesian Neural Networks for uncertainty quantification](https://mlsanity.com/paper/2012.02818/) (2020)
- [Dropout Injection at Test Time for Post Hoc Uncertainty Quantification in Neural Networks](https://mlsanity.com/paper/2302.02924/) (2023)
- [Adversarial Attack for Uncertainty Estimation: Identifying Critical Regions in Neural Networks](https://mlsanity.com/paper/2107.07618/) (2021)

## Built on by (most cited first, 10 of 10)

- [DAV-GSWT: Diffusion-Active-View Sampling for Data-Efficient Gaussian Splatting Wang Tiles](https://mlsanity.com/paper/2602.15355/) (2026)
- [PH-Dropout: Practical Epistemic Uncertainty Quantification for View Synthesis](https://mlsanity.com/paper/2410.05468/) (2024)
- [Epistemic Reject Option Prediction](https://mlsanity.com/paper/2511.04855/) (2025)
- [Unsupervised Source-Free Ranking of Biomedical Segmentation Models Under Distribution Shift](https://mlsanity.com/paper/2503.00450/) (2025)
- [EMoE: Training-Free Expert Disagreement for Uncertainty-Aware Text-to-Image Diffusion](https://mlsanity.com/paper/2505.13273/) (2025)
- [Robust Uncertainty Estimation under Distribution Shift via Difference Reconstruction](https://mlsanity.com/paper/2601.19341/) (2026)
- [FedVG: Gradient-Guided Aggregation for Enhanced Federated Learning](https://mlsanity.com/paper/2602.21399/) (2026)
- [CUPID: A Plug-in Framework for Joint Aleatoric and Epistemic Uncertainty Estimation with a Single Model](https://mlsanity.com/paper/2603.10745/) (2026)
- [Gradients with Respect to Semantics Preserving Embeddings Tell the Uncertainty of Large Language Models](https://mlsanity.com/paper/2605.04638/) (2026)
- [Perturbation-Based Epistemic Uncertainty for Failure Detection in Vision-Language-Action Models](https://mlsanity.com/paper/2606.20754/) (2026)

## Abstract (arXiv)

Epistemic uncertainty quantification (UQ) identifies where models lack knowledge. Traditional UQ methods, often based on Bayesian neural networks, are not suitable for pre-trained non-Bayesian models. Our study addresses quantifying epistemic uncertainty for any pre-trained model, which does not need the original training data or model modifications and can ensure broad applicability regardless of network architectures or training techniques. Specifically, we propose a gradient-based approach to assess epistemic uncertainty, analyzing the gradients of outputs relative to model parameters, and thereby indicating necessary model adjustments to accurately represent the inputs. We first explore theoretical guarantees of gradient-based methods for epistemic UQ, questioning the view that this uncertainty is only calculable through differences between multiple models. We further improve gradient-driven UQ by using class-specific weights for integrating gradients and emphasizing distinct contributions from neural network layers. Additionally, we enhance UQ accuracy by combining gradient and perturbation methods to refine the gradients. We evaluate our approach on out-of-distribution detection, uncertainty calibration, and active learning, demonstrating its superiority over current state-of-the-art UQ methods for pre-trained models.