# Detecting Underspecification with Local Ensembles

arXiv: https://arxiv.org/abs/1910.09573 · HTML: https://mlsanity.com/paper/1910.09573/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Oct 2019.

## Built on by (most cited first, 3 of 3)

- [GradOrth: A Simple yet Efficient Out-of-Distribution Detection with Orthogonal Projection of Gradients](https://mlsanity.com/paper/2308.00310/) (2023)
- [Uncertainty Quantification with the Empirical Neural Tangent Kernel](https://mlsanity.com/paper/2502.02870/) (2025)
- [Differentially Private Sampling from Rashomon Sets, and the Universality of Langevin Diffusion for Convex Optimization](https://mlsanity.com/paper/2204.01585/) (2022)

## Abstract (arXiv)

We present local ensembles, a method for detecting underspecification -- when many possible predictors are consistent with the training data and model class -- at test time in a pre-trained model. Our method uses local second-order information to approximate the variance of predictions across an ensemble of models from the same class. We compute this approximation by estimating the norm of the component of a test point's gradient that aligns with the low-curvature directions of the Hessian, and provide a tractable method for estimating this quantity. Experimentally, we show that our method is capable of detecting when a pre-trained model is underspecified on test data, with applications to out-of-distribution detection, detecting spurious correlates, and active learning.