# A Probabilistic Segment Anything Model for Ambiguity-Aware Medical Image Segmentation

arXiv: https://arxiv.org/abs/2509.05809 · HTML: https://mlsanity.com/paper/2509.05809/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 15 references that are in the map.
- First posted on arXiv 6 Sep 2025.

## Builds on (most cited first, 8 of 8)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [A Probabilistic U-Net for Segmentation of Ambiguous Images](https://mlsanity.com/paper/1806.05034/) (2018)
- [Trustworthy clinical AI solutions: a unified review of uncertainty quantification in deep learning models for medical image analysis](https://mlsanity.com/paper/2210.03736/) (2022)
- [Is segmentation uncertainty useful?](https://mlsanity.com/paper/2103.16265/) (2021)
- [AutoProSAM: Automated Prompting SAM for 3D Multi-Organ Segmentation](https://mlsanity.com/paper/2308.14936/) (2023)
- [Autoadaptive Medical Segment Anything Model](https://mlsanity.com/paper/2507.01828/) (2025)
- [Annotating Ambiguous Images: General Annotation Strategy for High-Quality Data with Real-World Biomedical Validation](https://mlsanity.com/paper/2306.12189/) (2023)
- [Detection of Breast Cancer Lumpectomy Margin with SAM-incorporated Forward-Forward Contrastive Learning](https://mlsanity.com/paper/2506.21006/) (2025)

## Built on by (most cited first, 1 of 1)

- [SARFA: Segment Anything with Radiomic Feature Alignment](https://mlsanity.com/paper/2607.13323/) (2026)

## Abstract (arXiv)

Recent advances in promptable segmentation, such as the Segment Anything Model (SAM), have enabled flexible, high-quality mask generation across a wide range of visual domains. However, SAM and similar models remain fundamentally deterministic, producing a single segmentation per object per prompt, and fail to capture the inherent ambiguity present in many real-world tasks. This limitation is particularly troublesome in medical imaging, where multiple plausible segmentations may exist due to annotation uncertainty or inter-expert variability. In this paper, we introduce Probabilistic SAM, a probabilistic extension of SAM that models a distribution over segmentations conditioned on both the input image and prompt. By incorporating a latent variable space and training with a variational objective, our model learns to generate diverse and plausible segmentation masks reflecting the variability in human annotations. The architecture integrates a prior and posterior network into the SAM framework, allowing latent codes to modulate the prompt embeddings during inference. The latent space allows for efficient sampling during inference, enabling uncertainty-aware outputs with minimal overhead. We evaluate Probabilistic SAM on the public LIDC-IDRI lung nodule dataset and demonstrate its ability to produce diverse outputs that align with expert disagreement, outperforming existing probabilistic baselines on uncertainty-aware metrics. Our code is available at: https://github.com/tbwa233/Probabilistic-SAM/.