# Probabilistic Modeling of Inter- and Intra-observer Variability in Medical Image Segmentation

arXiv: https://arxiv.org/abs/2307.11397 · HTML: https://mlsanity.com/paper/2307.11397/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 36 references that are in the map.
- First posted on arXiv 21 Jul 2023.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations](https://mlsanity.com/paper/1707.03237/) (2017)
- [Albumentations: fast and flexible image augmentations](https://mlsanity.com/paper/1809.06839/) (2018)
- [Variational Dropout and the Local Reparameterization Trick](https://mlsanity.com/paper/1506.02557/) (2015)
- [A Probabilistic U-Net for Segmentation of Ambiguous Images](https://mlsanity.com/paper/1806.05034/) (2018)
- [Learning From Noisy Labels By Regularized Estimation Of Annotator Confusion](https://mlsanity.com/paper/1902.03680/) (2019)
- [Going Deeper through the Gleason Scoring Scale: An Automatic end-to-end System for Histology Prostate Grading and Cribriform Pattern Detection](https://mlsanity.com/paper/2105.10490/) (2021)
- [Disentangling Human Error from the Ground Truth in Segmentation of Medical Images](https://mlsanity.com/paper/2007.15963/) (2020)
- [CryoNuSeg: A Dataset for Nuclei Instance Segmentation of Cryosectioned H&E-Stained Histological Images](https://mlsanity.com/paper/2101.00442/) (2021)
- [Probabilistic Attention based on Gaussian Processes for Deep Multiple Instance Learning](https://mlsanity.com/paper/2302.04061/) (2023)

## Built on by (most cited first, 17 of 17)

- [A review of uncertainty quantification in medical image analysis: probabilistic and non-probabilistic methods](https://mlsanity.com/paper/2310.06873/) (2023)
- [Tyche: Stochastic In-Context Learning for Medical Image Segmentation](https://mlsanity.com/paper/2401.13650/) (2024)
- [Diversified and Personalized Multi-rater Medical Image Segmentation](https://mlsanity.com/paper/2403.13417/) (2024)
- [A Review of Bayesian Uncertainty Quantification in Deep Probabilistic Image Segmentation](https://mlsanity.com/paper/2411.16370/) (2024)
- [Assessing Test-time Variability for Interactive 3D Medical Image Segmentation with Diverse Point Prompts](https://mlsanity.com/paper/2311.07806/) (2023)
- [DiffOSeg: Omni Medical Image Segmentation via Multi-Expert Collaboration Diffusion Model](https://mlsanity.com/paper/2507.13087/) (2025)
- [UnSegGNet: Unsupervised Image Segmentation using Graph Neural Networks](https://mlsanity.com/paper/2405.06057/) (2024)
- [RIDGE: Reproducibility, Integrity, Dependability, Generalizability, and Efficiency Assessment of Medical Image Segmentation Models](https://mlsanity.com/paper/2401.08847/) (2024)
- [Segmentation Strategies in Deep Learning for Prostate Cancer Diagnosis: A Comparative Study of Mamba, SAM, and YOLO](https://mlsanity.com/paper/2409.16205/) (2024)
- [Good Enough? An Investigation on the Impact of Label Quality in Large-Scale Medical Datasets](https://mlsanity.com/paper/2505.20928/) (2025)
- [Coreset selection based on Intra-class diversity](https://mlsanity.com/paper/2509.21380/) (2025)
- [ProSona: Prompt-Guided Personalization for Multi-Expert Medical Image Segmentation](https://mlsanity.com/paper/2511.08046/) (2025)
- [Probabilistic Modeling of Multi-rater Medical Image Segmentation for Diversity and Personalization](https://mlsanity.com/paper/2512.00748/) (2025)
- [Pancakes: Consistent Multi-Protocol Image Segmentation Across Biomedical Domains](https://mlsanity.com/paper/2512.13534/) (2025)
- [Harmonized Feature Conditioning and Frequency-Prompt Personalization for Multi-Rater Medical Segmentation](https://mlsanity.com/paper/2605.08210/) (2026)
- [Interpretable Probabilistic Medical Image Segmentation via Gaussian Process with Explicit Modelling of Annotation Bias and Variability](https://mlsanity.com/paper/2606.23177/) (2026)
- [Unsupervised Anatomical Feature Learning via Diffusion Models: Enhanced Medical Image Segmentation with Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2608.25693/) (2026)

## Abstract (arXiv)

Medical image segmentation is a challenging task, particularly due to inter- and intra-observer variability, even between medical experts. In this paper, we propose a novel model, called Probabilistic Inter-Observer and iNtra-Observer variation NetwOrk (Pionono). It captures the labeling behavior of each rater with a multidimensional probability distribution and integrates this information with the feature maps of the image to produce probabilistic segmentation predictions. The model is optimized by variational inference and can be trained end-to-end. It outperforms state-of-the-art models such as STAPLE, Probabilistic U-Net, and models based on confusion matrices. Additionally, Pionono predicts multiple coherent segmentation maps that mimic the rater's expert opinion, which provides additional valuable information for the diagnostic process. Experiments on real-world cancer segmentation datasets demonstrate the high accuracy and efficiency of Pionono, making it a powerful tool for medical image analysis.