# Ambiguous Medical Image Segmentation using Diffusion Models

arXiv: https://arxiv.org/abs/2304.04745 · HTML: https://mlsanity.com/paper/2304.04745/

## Key facts

- Cited by 92 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 39 of its 63 references that are in the map.
- First posted on arXiv 10 Apr 2023.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 30 of 39)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [RePaint: Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.09865/) (2022)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [Improved Techniques for Training Score-Based Generative Models](https://mlsanity.com/paper/2006.09011/) (2020)
- [ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2108.02938/) (2021)
- [Scaling Laws for Autoregressive Generative Modeling](https://mlsanity.com/paper/2010.14701/) (2020)
- [Label-Efficient Semantic Segmentation with Diffusion Models](https://mlsanity.com/paper/2112.03126/) (2021)
- [Medical Transformer: Gated Axial-Attention for Medical Image Segmentation](https://mlsanity.com/paper/2102.10662/) (2021)
- [A Probabilistic U-Net for Segmentation of Ambiguous Images](https://mlsanity.com/paper/1806.05034/) (2018)
- [SegDiff: Image Segmentation with Diffusion Probabilistic Models](https://mlsanity.com/paper/2112.00390/) (2021)
- [CE-Net: Context Encoder Network for 2D Medical Image Segmentation](https://mlsanity.com/paper/1903.02740/) (2019)
- [Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding](https://mlsanity.com/paper/1511.02680/) (2015)
- [ResUNet++: An Advanced Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1911.07067/) (2019)
- [Diffusion Models for Implicit Image Segmentation Ensembles](https://mlsanity.com/paper/2112.03145/) (2021)
- [UNeXt: MLP-based Rapid Medical Image Segmentation Network](https://mlsanity.com/paper/2203.04967/) (2022)
- [MedSegDiff: Medical Image Segmentation with Diffusion Probabilistic Model](https://mlsanity.com/paper/2211.00611/) (2022)
- [Transformation Consistent Self-ensembling Model for Semi-supervised Medical Image Segmentation](https://mlsanity.com/paper/1903.00348/) (2019)
- [PHiSeg: Capturing Uncertainty in Medical Image Segmentation](https://mlsanity.com/paper/1906.04045/) (2019)
- [Improving GANs Using Optimal Transport](https://mlsanity.com/paper/1803.05573/) (2018)
- [The Cramer Distance as a Solution to Biased Wasserstein Gradients](https://mlsanity.com/paper/1705.10743/) (2017)
- [Stochastic Segmentation Networks: Modelling Spatially Correlated Aleatoric Uncertainty](https://mlsanity.com/paper/2006.06015/) (2020)
- [A Hierarchical Probabilistic U-Net for Modeling Multi-Scale Ambiguities](https://mlsanity.com/paper/1905.13077/) (2019)

## Built on by (most cited first, 30 of 92)

- [A Comprehensive Survey of Foundation Models in Medicine](https://mlsanity.com/paper/2406.10729/) (2024)
- [AdaptiveSAM: Towards Efficient Tuning of SAM for Surgical Scene Segmentation](https://mlsanity.com/paper/2308.03726/) (2023)
- [Image Segmentation in Foundation Model Era: A Survey](https://mlsanity.com/paper/2408.12957/) (2024)
- [Contrastive Conditional Latent Diffusion for Audio-visual Segmentation](https://mlsanity.com/paper/2307.16579/) (2023)
- [Diffusion Model for Camouflaged Object Detection](https://mlsanity.com/paper/2308.00303/) (2023)
- [FlowSDF: Flow Matching for Medical Image Segmentation Using Distance Transforms](https://mlsanity.com/paper/2405.18087/) (2024)
- [DiffBoost: Enhancing Medical Image Segmentation via Text-Guided Diffusion Model](https://mlsanity.com/paper/2310.12868/) (2023)
- [CamoDiffusion: Camouflaged Object Detection via Conditional Diffusion Models](https://mlsanity.com/paper/2305.17932/) (2023)
- [GCD-DDPM: A Generative Change Detection Model Based on Difference-Feature Guided DDPM](https://mlsanity.com/paper/2306.03424/) (2023)
- [Tyche: Stochastic In-Context Learning for Medical Image Segmentation](https://mlsanity.com/paper/2401.13650/) (2024)
- [Treatment-aware Diffusion Probabilistic Model for Longitudinal MRI Generation and Diffuse Glioma Growth Prediction](https://mlsanity.com/paper/2309.05406/) (2023)
- [FreeSeg-Diff: Training-Free Open-Vocabulary Segmentation with Diffusion Models](https://mlsanity.com/paper/2403.20105/) (2024)
- [Generative AI-Driven Human Digital Twin in IoT-Healthcare: A Comprehensive Survey](https://mlsanity.com/paper/2401.13699/) (2024)
- [S-SAM: SVD-based Fine-Tuning of Segment Anything Model for Medical Image Segmentation](https://mlsanity.com/paper/2408.06447/) (2024)
- [Annotator Consensus Prediction for Medical Image Segmentation with Diffusion Models](https://mlsanity.com/paper/2306.09004/) (2023)
- [A Survey of Emerging Applications of Diffusion Probabilistic Models in MRI](https://mlsanity.com/paper/2311.11383/) (2023)
- [Diversified and Personalized Multi-rater Medical Image Segmentation](https://mlsanity.com/paper/2403.13417/) (2024)
- [Pi-fusion: Physics-informed diffusion model for learning fluid dynamics](https://mlsanity.com/paper/2406.03711/) (2024)
- [SimuScope: Realistic Endoscopic Synthetic Dataset Generation through Surgical Simulation and Diffusion Models](https://mlsanity.com/paper/2412.02332/) (2024)
- [A Review of Bayesian Uncertainty Quantification in Deep Probabilistic Image Segmentation](https://mlsanity.com/paper/2411.16370/) (2024)
- [MIFNet: Learning Modality-Invariant Features for Generalizable Multimodal Image Matching](https://mlsanity.com/paper/2501.11299/) (2025)
- [A Recycling Training Strategy for Medical Image Segmentation with Diffusion Denoising Models](https://mlsanity.com/paper/2308.16355/) (2023)
- [Conditional diffusion model with spatial attention and latent embedding for medical image segmentation](https://mlsanity.com/paper/2502.06997/) (2025)
- [DiffAtlas: GenAI-fying Atlas Segmentation via Image-Mask Diffusion](https://mlsanity.com/paper/2503.06748/) (2025)
- [Aleatoric Uncertainty Medical Image Segmentation Estimation via Flow Matching](https://mlsanity.com/paper/2507.22418/) (2025)
- [Ambiguous Medical Image Segmentation Using Diffusion Schr"odinger Bridge](https://mlsanity.com/paper/2509.17187/) (2025)
- [C-DARL: Contrastive diffusion adversarial representation learning for label-free blood vessel segmentation](https://mlsanity.com/paper/2308.00193/) (2023)
- [Prompt-Softbox-Prompt: A Free-Text Embedding Control for Image Editing](https://mlsanity.com/paper/2408.13623/) (2024)
- [RadGazeGen: Radiomics and Gaze-guided Medical Image Generation using Diffusion Models](https://mlsanity.com/paper/2410.00307/) (2024)
- [BrainMRDiff: A Diffusion Model for Anatomically Consistent Brain MRI Synthesis](https://mlsanity.com/paper/2504.04532/) (2025)

## Abstract (arXiv)

Collective insights from a group of experts have always proven to outperform an individual's best diagnostic for clinical tasks. For the task of medical image segmentation, existing research on AI-based alternatives focuses more on developing models that can imitate the best individual rather than harnessing the power of expert groups. In this paper, we introduce a single diffusion model-based approach that produces multiple plausible outputs by learning a distribution over group insights. Our proposed model generates a distribution of segmentation masks by leveraging the inherent stochastic sampling process of diffusion using only minimal additional learning. We demonstrate on three different medical image modalities- CT, ultrasound, and MRI that our model is capable of producing several possible variants while capturing the frequencies of their occurrences. Comprehensive results show that our proposed approach outperforms existing state-of-the-art ambiguous segmentation networks in terms of accuracy while preserving naturally occurring variation. We also propose a new metric to evaluate the diversity as well as the accuracy of segmentation predictions that aligns with the interest of clinical practice of collective insights.