# Bayesian Uncertainty and Expected Gradient Length -- Regression: Two Sides Of The Same Coin?

arXiv: https://arxiv.org/abs/2104.09493 · HTML: https://mlsanity.com/paper/2104.09493/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 56 references that are in the map.
- First posted on arXiv 19 Apr 2021.

## Builds on (most cited first, 17 of 17)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [Learning Loss for Active Learning](https://mlsanity.com/paper/1905.03677/) (2019)
- [DeepPose: Human Pose Estimation via Deep Neural Networks](https://mlsanity.com/paper/1312.4659/) (2013)
- [Adversarial Sampling for Active Learning](https://mlsanity.com/paper/1808.06671/) (2018)
- [Efficient Per-Example Gradient Computations](https://mlsanity.com/paper/1510.01799/) (2015)
- [Active Learning for Speech Recognition: the Power of Gradients](https://mlsanity.com/paper/1612.03226/) (2016)
- [Toward fast and accurate human pose estimation via soft-gated skip connections](https://mlsanity.com/paper/2002.11098/) (2020)
- [Asymptotic Analysis of Objectives based on Fisher Information in Active Learning](https://mlsanity.com/paper/1605.08798/) (2016)
- [Active Learning for Bayesian 3D Hand Pose Estimation](https://mlsanity.com/paper/2010.00694/) (2020)
- [BaCOUn: Bayesian Classifers with Out-of-Distribution Uncertainty](https://mlsanity.com/paper/2007.06096/) (2020)
- [LEt-SNE: A Hybrid Approach To Data Embedding and Visualization of Hyperspectral Imagery](https://mlsanity.com/paper/1910.08790/) (2019)

## Built on by (most cited first, 3 of 3)

- [VL4Pose: Active Learning Through Out-Of-Distribution Detection For Pose Estimation](https://mlsanity.com/paper/2210.06028/) (2022)
- [TIC-TAC: A Framework for Improved Covariance Estimation in Deep Heteroscedastic Regression](https://mlsanity.com/paper/2310.18953/) (2023)
- [Transformer-Based Active Learning for Data-Efficient Vaccine Epitope Selection in PRRS](https://mlsanity.com/paper/2606.28659/) (2026)

## Abstract (arXiv)

Active learning algorithms select a subset of data for annotation to maximize the model performance on a budget. One such algorithm is Expected Gradient Length, which as the name suggests uses the approximate gradient induced per example in the sampling process. While Expected Gradient Length has been successfully used for classification and regression, the formulation for regression remains intuitively driven. Hence, our theoretical contribution involves deriving this formulation, thereby supporting the experimental evidence. Subsequently, we show that expected gradient length in regression is equivalent to Bayesian uncertainty. If certain assumptions are infeasible, our algorithmic contribution (EGL++) approximates the effect of ensembles with a single deterministic network. Instead of computing multiple possible inferences per input, we leverage previously annotated samples to quantify the probability of previous labels being the true label. Such an approach allows us to extend expected gradient length to a new task: human pose estimation. We perform experimental validation on two human pose datasets (MPII and LSP/LSPET), highlighting the interpretability and competitiveness of EGL++ with different active learning algorithms for human pose estimation.