# Do you understand epistemic uncertainty? Think again! Rigorous frequentist epistemic uncertainty estimation in regression

arXiv: https://arxiv.org/abs/2503.13317 · HTML: https://mlsanity.com/paper/2503.13317/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 17 references that are in the map.
- First posted on arXiv 17 Mar 2025.

## Builds on (most cited first, 11 of 11)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [What Are Bayesian Neural Network Posteriors Really Like?](https://mlsanity.com/paper/2104.14421/) (2021)
- [On the Pitfalls of Heteroscedastic Uncertainty Estimation with Probabilistic Neural Networks](https://mlsanity.com/paper/2203.09168/) (2022)
- [Distribution Calibration for Regression](https://mlsanity.com/paper/1905.06023/) (2019)
- [Evaluating and Calibrating Uncertainty Prediction in Regression Tasks](https://mlsanity.com/paper/1905.11659/) (2019)
- [Benchmarking Uncertainty Disentanglement: Specialized Uncertainties for Specialized Tasks](https://mlsanity.com/paper/2402.19460/) (2024)
- [On Second-Order Scoring Rules for Epistemic Uncertainty Quantification](https://mlsanity.com/paper/2301.12736/) (2023)
- [A Rigorous Link between Deep Ensembles and (Variational) Bayesian Methods](https://mlsanity.com/paper/2305.15027/) (2023)
- [Experts Don't Cheat: Learning What You Don't Know By Predicting Pairs](https://mlsanity.com/paper/2402.08733/) (2024)
- [ZigZag: Universal Sampling-free Uncertainty Estimation Through Two-Step Inference](https://mlsanity.com/paper/2211.11435/) (2022)

## Abstract (arXiv)

Quantifying model uncertainty is critical for understanding prediction reliability, yet distinguishing between aleatoric and epistemic uncertainty remains challenging. We extend recent work from classification to regression to provide a novel frequentist approach to epistemic and aleatoric uncertainty estimation. We train models to generate conditional predictions by feeding their initial output back as an additional input. This method allows for a rigorous measurement of model uncertainty by observing how prediction responses change when conditioned on the model's previous answer. We provide a complete theoretical framework to analyze epistemic uncertainty in regression in a frequentist way, and explain how it can be exploited in practice to gauge a model's uncertainty, with minimal changes to the original architecture.