# Localising Dropout Variance in Twin Networks

arXiv: https://arxiv.org/abs/2507.03622 · HTML: https://mlsanity.com/paper/2507.03622/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 14 references that are in the map.
- First posted on arXiv 4 Jul 2025.

## Builds on (most cited first, 10 of 10)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [Meta-learners for Estimating Heterogeneous Treatment Effects using Machine Learning](https://mlsanity.com/paper/1706.03461/) (2017)
- [Adapting Neural Networks for the Estimation of Treatment Effects](https://mlsanity.com/paper/1906.02120/) (2019)
- [Estimating Epistemic and Aleatoric Uncertainty with a Single Model](https://mlsanity.com/paper/2402.03478/) (2024)
- [Rethinking Aleatoric and Epistemic Uncertainty](https://mlsanity.com/paper/2412.20892/) (2024)
- [From Aleatoric to Epistemic: Exploring Uncertainty Quantification Techniques in Artificial Intelligence](https://mlsanity.com/paper/2501.03282/) (2025)
- [A Survey of Deep Causal Models and Their Industrial Applications](https://mlsanity.com/paper/2209.08860/) (2022)

## Abstract (arXiv)

Accurate individual treatment-effect estimation demands not only reliable point predictions but also uncertainty measures that help practitioners locate the source of model failure. We introduce a layer-wise variance decomposition for deep twin-network models: by toggling Monte Carlo Dropout independently in the shared encoder and the outcome heads, we split total predictive variance into an encoder component (σenc2) and a head component (σhead2), with σenc2 + σhead2 ≈ σtot2 by the law of total variance. Across three synthetic covariate-shift regimes, the encoder component dominates under distributional shift (ρenc=0.53) while the head component becomes informative only once encoder uncertainty is controlled. On a real-world twins cohort with induced multivariate shift, only σenc2 spikes on out-of-distribution samples and becomes the primary error predictor (ρenc!≈!0.89), while σhead2 remains flat. The decomposition adds negligible cost over standard MC Dropout and provides a practical diagnostic for deciding whether to collect more diverse covariates or more outcome data.