# Depth Uncertainty in Neural Networks

arXiv: https://arxiv.org/abs/2006.08437 · HTML: https://mlsanity.com/paper/2006.08437/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jun 2020.

## Built on by (most cited first, 30 of 57)

- [Trusted Multi-View Classification with Dynamic Evidential Fusion](https://mlsanity.com/paper/2204.11423/) (2022)
- [To Believe or Not to Believe Your LLM](https://mlsanity.com/paper/2406.02543/) (2024)
- [TabM: Advancing Tabular Deep Learning with Parameter-Efficient Ensembling](https://mlsanity.com/paper/2410.24210/) (2024)
- [DEUP: Direct Epistemic Uncertainty Prediction](https://mlsanity.com/paper/2102.08501/) (2021)
- [Trustworthy Multimodal Regression with Mixture of Normal-inverse Gamma Distributions](https://mlsanity.com/paper/2111.08456/) (2021)
- [UMIX: Improving Importance Weighting for Subpopulation Shift via Uncertainty-Aware Mixup](https://mlsanity.com/paper/2209.08928/) (2022)
- [Adapting the Linearised Laplace Model Evidence for Modern Deep Learning](https://mlsanity.com/paper/2206.08900/) (2022)
- [Learning Layer-wise Equivariances Automatically using Gradients](https://mlsanity.com/paper/2310.06131/) (2023)
- [Generative Uncertainty in Diffusion Models](https://mlsanity.com/paper/2502.20946/) (2025)
- [Sparse MoEs meet Efficient Ensembles](https://mlsanity.com/paper/2110.03360/) (2021)
- [Sampling-based inference for large linear models, with application to linearised Laplace](https://mlsanity.com/paper/2210.04994/) (2022)
- [DebSDF: Delving into the Details and Bias of Neural Indoor Scene Reconstruction](https://mlsanity.com/paper/2308.15536/) (2023)
- [Towards Anytime Classification in Early-Exit Architectures by Enforcing Conditional Monotonicity](https://mlsanity.com/paper/2306.02652/) (2023)
- [Blurs Behave Like Ensembles: Spatial Smoothings to Improve Accuracy, Uncertainty, and Robustness](https://mlsanity.com/paper/2105.12639/) (2021)
- [Enabling Uncertainty Estimation in Iterative Neural Networks](https://mlsanity.com/paper/2403.16732/) (2024)
- [Uncertainty Estimation for Computed Tomography with a Linearised Deep Image Prior](https://mlsanity.com/paper/2203.00479/) (2022)
- [Variational Inference for Infinitely Deep Neural Networks](https://mlsanity.com/paper/2209.10091/) (2022)
- [Deep Classifiers with Label Noise Modeling and Distance Awareness](https://mlsanity.com/paper/2110.02609/) (2021)
- [Improving robustness and calibration in ensembles with diversity regularization](https://mlsanity.com/paper/2201.10908/) (2022)
- [Bayesian Experimental Design for Computed Tomography with the Linearised Deep Image Prior](https://mlsanity.com/paper/2207.05714/) (2022)
- [Probabilistic Circuits That Know What They Don't Know](https://mlsanity.com/paper/2302.06544/) (2023)
- [Model-Value Inconsistency as a Signal for Epistemic Uncertainty](https://mlsanity.com/paper/2112.04153/) (2021)
- [Ensembling over Classifiers: a Bias-Variance Perspective](https://mlsanity.com/paper/2206.10566/) (2022)
- [Early-Exit Neural Networks with Nested Prediction Sets](https://mlsanity.com/paper/2311.05931/) (2023)
- [Ex uno plures: Splitting One Model into an Ensemble of Subnetworks](https://mlsanity.com/paper/2106.04767/) (2021)
- [Uncertainty Estimation for 3D Object Detection via Evidential Learning](https://mlsanity.com/paper/2410.23910/) (2024)
- [Integrating Large Pre-trained Models into Multimodal Named Entity Recognition with Evidential Fusion](https://mlsanity.com/paper/2306.16991/) (2023)
- [QUTE: Quantifying Uncertainty in TinyML with Early-exit-assisted ensembles for model-monitoring](https://mlsanity.com/paper/2404.12599/) (2024)
- [Transitional Uncertainty with Layered Intermediate Predictions](https://mlsanity.com/paper/2405.17494/) (2024)
- [Addressing Bias in Active Learning with Depth Uncertainty Networks... or Not](https://mlsanity.com/paper/2112.06926/) (2021)

## Abstract (arXiv)

Existing methods for estimating uncertainty in deep learning tend to require multiple forward passes, making them unsuitable for applications where computational resources are limited. To solve this, we perform probabilistic reasoning over the depth of neural networks. Different depths correspond to subnetworks which share weights and whose predictions are combined via marginalisation, yielding model uncertainty. By exploiting the sequential structure of feed-forward networks, we are able to both evaluate our training objective and make predictions with a single forward pass. We validate our approach on real-world regression and image classification tasks. Our approach provides uncertainty calibration, robustness to dataset shift, and accuracies competitive with more computationally expensive baselines.