# Single Model Uncertainty Estimation via Stochastic Data Centering

arXiv: https://arxiv.org/abs/2207.07235 · HTML: https://mlsanity.com/paper/2207.07235/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 53 references that are in the map.
- First posted on arXiv 14 Jul 2022.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Energy-based Out-of-distribution Detection](https://mlsanity.com/paper/2010.03759/) (2020)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent](https://mlsanity.com/paper/1902.06720/) (2019)
- [Deep Neural Networks as Gaussian Processes](https://mlsanity.com/paper/1711.00165/) (2017)
- [Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks](https://mlsanity.com/paper/1901.08584/) (2019)
- [Bayesian Deep Learning and a Probabilistic Perspective of Generalization](https://mlsanity.com/paper/2002.08791/) (2020)
- [Deep Ensembles: A Loss Landscape Perspective](https://mlsanity.com/paper/1912.02757/) (2019)
- [Gaussian Process Behaviour in Wide Deep Neural Networks](https://mlsanity.com/paper/1804.11271/) (2018)
- [A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions](https://mlsanity.com/paper/2006.02903/) (2020)
- [On the Inductive Bias of Neural Tangent Kernels](https://mlsanity.com/paper/1905.12173/) (2019)
- [Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes](https://mlsanity.com/paper/1810.05148/) (2018)
- [Improving model calibration with accuracy versus uncertainty optimization](https://mlsanity.com/paper/2012.07923/) (2020)
- [DEUP: Direct Epistemic Uncertainty Prediction](https://mlsanity.com/paper/2102.08501/) (2021)
- [Bayesian Deep Ensembles via the Neural Tangent Kernel](https://mlsanity.com/paper/2007.05864/) (2020)
- [Uncertainty Baselines: Benchmarks for Uncertainty & Robustness in Deep Learning](https://mlsanity.com/paper/2106.04015/) (2021)
- [Financial Applications of Gaussian Processes and Bayesian Optimization](https://mlsanity.com/paper/1903.04841/) (2019)
- [A Spectral Approach for the Design of Experiments: Design, Analysis and Algorithms](https://mlsanity.com/paper/1712.06028/) (2017)

## Built on by (most cited first, 20 of 20)

- [A Data-Driven Measure of Relative Uncertainty for Misclassification Detection](https://mlsanity.com/paper/2306.01710/) (2023)
- [Accurate and Scalable Estimation of Epistemic Uncertainty for Graph Neural Networks](https://mlsanity.com/paper/2401.03350/) (2024)
- [Uncertainty Quantification on Graph Learning: A Survey](https://mlsanity.com/paper/2404.14642/) (2024)
- [Transitional Uncertainty with Layered Intermediate Predictions](https://mlsanity.com/paper/2405.17494/) (2024)
- [Watch Your Head: Assembling Projection Heads to Save the Reliability of Federated Models](https://mlsanity.com/paper/2402.16255/) (2024)
- [Uncertainty-Aware Deep Neural Representations for Visual Analysis of Vector Field Data](https://mlsanity.com/paper/2407.16119/) (2024)
- [Uncertainty-Informed Volume Visualization using Implicit Neural Representation](https://mlsanity.com/paper/2408.06018/) (2024)
- [Uncertainty Estimation and Out-of-Distribution Detection for Deep Learning-Based Image Reconstruction using the Local Lipschitz](https://mlsanity.com/paper/2305.07618/) (2023)
- [Visual Analysis of Prediction Uncertainty in Neural Networks for Deep Image Synthesis](https://mlsanity.com/paper/2406.18545/) (2024)
- [Uncertainty Estimation for Heterophilic Graphs Through the Lens of Information Theory](https://mlsanity.com/paper/2505.22152/) (2025)
- [Uncertainty Estimation on Graphs with Structure Informed Stochastic Partial Differential Equations](https://mlsanity.com/paper/2506.06907/) (2025)
- [On the Use of Anchoring for Training Vision Models](https://mlsanity.com/paper/2406.00529/) (2024)
- [Scalable Uncertainty Quantification for Extreme Weather Forecasting via Empirical Neural Tangent Kernels](https://mlsanity.com/paper/2606.02886/) (2026)
- [Accurate and Scalable Estimation of Epistemic Uncertainty for Graph Neural Networks](https://mlsanity.com/paper/2309.10976/) (2023)
- [PAGER: A Framework for Failure Analysis of Deep Regression Models](https://mlsanity.com/paper/2309.10977/) (2023)
- [LLM-Assisted Red Teaming of Diffusion Models through "Failures Are Fated, But Can Be Faded"](https://mlsanity.com/paper/2410.16738/) (2024)
- [Soft Checksums to Flag Untrustworthy Machine Learning Surrogate Predictions and Application to Atomic Physics Simulations](https://mlsanity.com/paper/2412.03497/) (2024)
- [A Framework for Uncertainty Quantification Based on Nearest Neighbors Across Layers](https://mlsanity.com/paper/2506.19895/) (2025)
- [Information Theory in Open-world Machine Learning Foundations, Frameworks, and Future Direction](https://mlsanity.com/paper/2510.15422/) (2025)
- [Spiking Graph Predictive Coding for Reliable OOD Generalization](https://mlsanity.com/paper/2602.19392/) (2026)

## Abstract (arXiv)

We are interested in estimating the uncertainties of deep neural networks, which play an important role in many scientific and engineering problems. In this paper, we present a striking new finding that an ensemble of neural networks with the same weight initialization, trained on datasets that are shifted by a constant bias gives rise to slightly inconsistent trained models, where the differences in predictions are a strong indicator of epistemic uncertainties. Using the neural tangent kernel (NTK), we demonstrate that this phenomena occurs in part because the NTK is not shift-invariant. Since this is achieved via a trivial input transformation, we show that this behavior can therefore be approximated by training a single neural network -- using a technique that we call Delta-UQ -- that estimates uncertainty around prediction by marginalizing out the effect of the biases during inference. We show that Delta-UQ's uncertainty estimates are superior to many of the current methods on a variety of benchmarks -- outlier rejection, calibration under distribution shift, and sequential design optimization of black box functions. Code for Delta-UQ can be accessed at https://github.com/LLNL/DeltaUQ