# Variational Inference for Quantifying Inter-observer Variability in Segmentation of Anatomical Structures

arXiv: https://arxiv.org/abs/2201.07106 · HTML: https://mlsanity.com/paper/2201.07106/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 26 references that are in the map.
- First posted on arXiv 18 Jan 2022.

## Builds on (most cited first, 17 of 17)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [A Probabilistic U-Net for Segmentation of Ambiguous Images](https://mlsanity.com/paper/1806.05034/) (2018)
- [PHiSeg: Capturing Uncertainty in Medical Image Segmentation](https://mlsanity.com/paper/1906.04045/) (2019)
- [Stochastic Segmentation Networks: Modelling Spatially Correlated Aleatoric Uncertainty](https://mlsanity.com/paper/2006.06015/) (2020)
- [A Hierarchical Probabilistic U-Net for Modeling Multi-Scale Ambiguities](https://mlsanity.com/paper/1905.13077/) (2019)
- [Supervised Uncertainty Quantification for Segmentation with Multiple Annotations](https://mlsanity.com/paper/1907.01949/) (2019)
- [Adapting Off-the-Shelf Source Segmenter for Target Medical Image Segmentation](https://mlsanity.com/paper/2106.12497/) (2021)
- [Importance-Aware Semantic Segmentation in Self-Driving with Discrete Wasserstein Training](https://mlsanity.com/paper/2010.12440/) (2020)
- [Tutorial: Deriving the Standard Variational Autoencoder (VAE) Loss Function](https://mlsanity.com/paper/1907.08956/) (2019)
- [A Unified Conditional Disentanglement Framework for Multimodal Brain MR Image Translation](https://mlsanity.com/paper/2101.05434/) (2021)
- [Generative Self-training for Cross-domain Unsupervised Tagged-to-Cine MRI Synthesis](https://mlsanity.com/paper/2106.12499/) (2021)
- [Energy-constrained Self-training for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2101.00316/) (2021)
- [AUTO3D: Novel view synthesis through unsupervisely learned variational viewpoint and global 3D representation](https://mlsanity.com/paper/2007.06620/) (2020)
- [Self-semantic contour adaptation for cross modality brain tumor segmentation](https://mlsanity.com/paper/2201.05022/) (2022)
- [Segmentation of Cardiac Structures via Successive Subspace Learning with Saab Transform from Cine MRI](https://mlsanity.com/paper/2107.10718/) (2021)
- [Reinforced Wasserstein Training for Severity-Aware Semantic Segmentation in Autonomous Driving](https://mlsanity.com/paper/2008.04751/) (2020)

## Built on by (most cited first, 4 of 4)

- [Deep Unsupervised Domain Adaptation: A Review of Recent Advances and Perspectives](https://mlsanity.com/paper/2208.07422/) (2022)
- [A Review of Bayesian Uncertainty Quantification in Deep Probabilistic Image Segmentation](https://mlsanity.com/paper/2411.16370/) (2024)
- [Mixture-of-Shape-Experts (MoSE): End-to-End Shape Dictionary Framework to Prompt SAM for Generalizable Medical Segmentation](https://mlsanity.com/paper/2504.09601/) (2025)
- [Quantification of Uncertainty with Adversarial Models in Medical Image Segmentation](https://mlsanity.com/paper/2606.18860/) (2026)

## Abstract (arXiv)

Lesions or organ boundaries visible through medical imaging data are often ambiguous, thus resulting in significant variations in multi-reader delineations, i.e., the source of aleatoric uncertainty. In particular, quantifying the inter-observer variability of manual annotations with Magnetic Resonance (MR) Imaging data plays a crucial role in establishing a reference standard for various diagnosis and treatment tasks. Most segmentation methods, however, simply model a mapping from an image to its single segmentation map and do not take the disagreement of annotators into consideration. In order to account for inter-observer variability, without sacrificing accuracy, we propose a novel variational inference framework to model the distribution of plausible segmentation maps, given a specific MR image, which explicitly represents the multi-reader variability. Specifically, we resort to a latent vector to encode the multi-reader variability and counteract the inherent information loss in the imaging data. Then, we apply a variational autoencoder network and optimize its evidence lower bound (ELBO) to efficiently approximate the distribution of the segmentation map, given an MR image. Experimental results, carried out with the QUBIQ brain growth MRI segmentation datasets with seven annotators, demonstrate the effectiveness of our approach.