# Modeling Annotator Preference and Stochastic Annotation Error for Medical Image Segmentation

arXiv: https://arxiv.org/abs/2111.13410 · HTML: https://mlsanity.com/paper/2111.13410/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 47 references that are in the map.
- First posted on arXiv 26 Nov 2021.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [FDA: Fourier Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2004.05498/) (2020)
- [A Probabilistic U-Net for Segmentation of Ambiguous Images](https://mlsanity.com/paper/1806.05034/) (2018)
- [PHiSeg: Capturing Uncertainty in Medical Image Segmentation](https://mlsanity.com/paper/1906.04045/) (2019)
- [Learning From Noisy Labels By Regularized Estimation Of Annotator Confusion](https://mlsanity.com/paper/1902.03680/) (2019)
- [MS-Net: Multi-Site Network for Improving Prostate Segmentation with Heterogeneous MRI Data](https://mlsanity.com/paper/2002.03366/) (2020)
- [Stochastic Segmentation Networks: Modelling Spatially Correlated Aleatoric Uncertainty](https://mlsanity.com/paper/2006.06015/) (2020)
- [DoDNet: Learning to segment multi-organ and tumors from multiple partially labeled datasets](https://mlsanity.com/paper/2011.10217/) (2020)
- [Who Said What: Modeling Individual Labelers Improves Classification](https://mlsanity.com/paper/1703.08774/) (2017)
- [A Hierarchical Probabilistic U-Net for Modeling Multi-Scale Ambiguities](https://mlsanity.com/paper/1905.13077/) (2019)
- [Disentangling Human Error from the Ground Truth in Segmentation of Medical Images](https://mlsanity.com/paper/2007.15963/) (2020)
- [On the Effect of Inter-observer Variability for a Reliable Estimation of Uncertainty of Medical Image Segmentation](https://mlsanity.com/paper/1806.02562/) (2018)
- [Supervised Uncertainty Quantification for Segmentation with Multiple Annotations](https://mlsanity.com/paper/1907.01949/) (2019)
- [Learning to Segment Skin Lesions from Noisy Annotations](https://mlsanity.com/paper/1906.03815/) (2019)
- [An Empirical Study into Annotator Agreement, Ground Truth Estimation, and Algorithm Evaluation](https://mlsanity.com/paper/1307.0426/) (2013)
- [D-LEMA: Deep Learning Ensembles from Multiple Annotations -- Application to Skin Lesion Segmentation](https://mlsanity.com/paper/2012.07206/) (2020)
- [Learning from Ambiguous Labels for Lung Nodule Malignancy Prediction](https://mlsanity.com/paper/2104.11436/) (2021)
- [Calibrated Adversarial Refinement for Stochastic Semantic Segmentation](https://mlsanity.com/paper/2006.13144/) (2020)
- [Handling Inter-Annotator Agreement for Automated Skin Lesion Segmentation](https://mlsanity.com/paper/1906.02415/) (2019)
- [Impact of individual rater style on deep learning uncertainty in medical imaging segmentation](https://mlsanity.com/paper/2105.02197/) (2021)
- [Cross-denoising Network against Corrupted Labels in Medical Image Segmentation with Domain Shift](https://mlsanity.com/paper/2006.10990/) (2020)
- [Pathological Image Segmentation with Noisy Labels](https://mlsanity.com/paper/2104.02602/) (2021)

## Built on by (most cited first, 12 of 12)

- [Rethinking Generalization: The Impact of Annotation Style on Medical Image Segmentation](https://mlsanity.com/paper/2210.17398/) (2022)
- [Federated Learning for Medical Image Classification: A Comprehensive Benchmark](https://mlsanity.com/paper/2504.05238/) (2025)
- [Towards Clinician-Preferred Segmentation: Leveraging Human-in-the-Loop for Test Time Adaptation in Medical Image Segmentation](https://mlsanity.com/paper/2405.08270/) (2024)
- [Autoregressive Medical Image Segmentation via Next-Scale Mask Prediction](https://mlsanity.com/paper/2502.20784/) (2025)
- [URL: Combating Label Noise for Lung Nodule Malignancy Grading](https://mlsanity.com/paper/2308.08772/) (2023)
- [DiffOSeg: Omni Medical Image Segmentation via Multi-Expert Collaboration Diffusion Model](https://mlsanity.com/paper/2507.13087/) (2025)
- [LiMT: A Multi-task Liver Image Benchmark Dataset](https://mlsanity.com/paper/2511.19889/) (2025)
- [Beyond Manual Annotation: A Human-AI Collaborative Framework for Medical Image Segmentation Using Only "Better or Worse" Expert Feedback](https://mlsanity.com/paper/2507.05815/) (2025)
- [Learning from Disagreement: A Group Decision Simulation Framework for Robust Medical Image Segmentation](https://mlsanity.com/paper/2510.10462/) (2025)
- [Attention-Based Prototype Calibration for Multi-Rater Few-Shot Medical Image Segmentation](https://mlsanity.com/paper/2606.16325/) (2026)
- [Federated Medical Image Segmentation under Real-World Label Noise: A Benchmark Suite for Noisy Label Learning Method Selection](https://mlsanity.com/paper/2606.16868/) (2026)
- [Decoupled Single-Mask Annotation Noise Detection via Cross-Sectional Patch Self-Consistency](https://mlsanity.com/paper/2607.05965/) (2026)

## Abstract (arXiv)

Manual annotation of medical images is highly subjective, leading to inevitable and huge annotation biases. Deep learning models may surpass human performance on a variety of tasks, but they may also mimic or amplify these biases. Although we can have multiple annotators and fuse their annotations to reduce stochastic errors, we cannot use this strategy to handle the bias caused by annotators' preferences. In this paper, we highlight the issue of annotator-related biases on medical image segmentation tasks, and propose a Preference-involved Annotation Distribution Learning (PADL) framework to address it from the perspective of disentangling an annotator's preference from stochastic errors using distribution learning so as to produce not only a meta segmentation but also the segmentation possibly made by each annotator. Under this framework, a stochastic error modeling (SEM) module estimates the meta segmentation map and average stochastic error map, and a series of human preference modeling (HPM) modules estimate each annotator's segmentation and the corresponding stochastic error. We evaluated our PADL framework on two medical image benchmarks with different imaging modalities, which have been annotated by multiple medical professionals, and achieved promising performance on all five medical image segmentation tasks.