# How inter-rater variability relates to aleatoric and epistemic uncertainty: a case study with deep learning-based paraspinal muscle segmentation

arXiv: https://arxiv.org/abs/2308.06964 · HTML: https://mlsanity.com/paper/2308.06964/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 21 references that are in the map.
- First posted on arXiv 14 Aug 2023.

## Builds on (most cited first, 14 of 14)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [Bayesian Deep Learning and a Probabilistic Perspective of Generalization](https://mlsanity.com/paper/2002.08791/) (2020)
- [Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks](https://mlsanity.com/paper/1807.07356/) (2018)
- [DropConnect Is Effective in Modeling Uncertainty of Bayesian Deep Networks](https://mlsanity.com/paper/1906.04569/) (2019)
- [Recalibration of Aleatoric and Epistemic Regression Uncertainty in Medical Imaging](https://mlsanity.com/paper/2104.12376/) (2021)
- [Label fusion and training methods for reliable representation of inter-rater uncertainty](https://mlsanity.com/paper/2202.07550/) (2022)
- [Characterizing Sources of Uncertainty to Proxy Calibration and Disambiguate Annotator and Data Bias](https://mlsanity.com/paper/1909.09285/) (2019)
- [Rethinking Generalization: The Impact of Annotation Style on Medical Image Segmentation](https://mlsanity.com/paper/2210.17398/) (2022)
- [A Quantitative Comparison of Epistemic Uncertainty Maps Applied to Multi-Class Segmentation](https://mlsanity.com/paper/2109.10702/) (2021)
- [Impact of individual rater style on deep learning uncertainty in medical imaging segmentation](https://mlsanity.com/paper/2105.02197/) (2021)

## Built on by (most cited first, 4 of 4)

- [A Review of Bayesian Uncertainty Quantification in Deep Probabilistic Image Segmentation](https://mlsanity.com/paper/2411.16370/) (2024)
- [Reliability of deep learning models for anatomical landmark detection: The role of inter-rater variability](https://mlsanity.com/paper/2411.17850/) (2024)
- [Quantification of Uncertainty with Adversarial Models in Medical Image Segmentation](https://mlsanity.com/paper/2606.18860/) (2026)
- [Unsupervised Anatomical Feature Learning via Diffusion Models: Enhanced Medical Image Segmentation with Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2608.25693/) (2026)

## Abstract (arXiv)

Recent developments in deep learning (DL) techniques have led to great performance improvement in medical image segmentation tasks, especially with the latest Transformer model and its variants. While labels from fusing multi-rater manual segmentations are often employed as ideal ground truths in DL model training, inter-rater variability due to factors such as training bias, image noise, and extreme anatomical variability can still affect the performance and uncertainty of the resulting algorithms. Knowledge regarding how inter-rater variability affects the reliability of the resulting DL algorithms, a key element in clinical deployment, can help inform better training data construction and DL models, but has not been explored extensively. In this paper, we measure aleatoric and epistemic uncertainties using test-time augmentation (TTA), test-time dropout (TTD), and deep ensemble to explore their relationship with inter-rater variability. Furthermore, we compare UNet and TransUNet to study the impacts of Transformers on model uncertainty with two label fusion strategies. We conduct a case study using multi-class paraspinal muscle segmentation from T2w MRIs. Our study reveals the interplay between inter-rater variability and uncertainties, affected by choices of label fusion strategies and DL models.