# Mask-conditioned latent diffusion for generating gastrointestinal polyp images

arXiv: https://arxiv.org/abs/2304.05233 · HTML: https://mlsanity.com/paper/2304.05233/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 21 of its 44 references that are in the map.
- First posted on arXiv 11 Apr 2023.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations](https://mlsanity.com/paper/1707.03237/) (2017)
- [Generative Adversarial Networks: An Overview](https://mlsanity.com/paper/1710.07035/) (2017)
- [Kvasir-SEG: A Segmented Polyp Dataset](https://mlsanity.com/paper/1911.07069/) (2019)
- [Image inpainting: A review](https://mlsanity.com/paper/1909.06399/) (2019)
- [Synthetic Data -- what, why and how?](https://mlsanity.com/paper/2205.03257/) (2022)
- [SinGAN-Seg: Synthetic training data generation for medical image segmentation](https://mlsanity.com/paper/2107.00471/) (2021)
- [An Extensive Study on Cross-Dataset Bias and Evaluation Metrics Interpretation for Machine Learning applied to Gastrointestinal Tract Abnormality Classification](https://mlsanity.com/paper/2005.03912/) (2020)
- [PolypConnect: Image inpainting for generating realistic gastrointestinal tract images with polyps](https://mlsanity.com/paper/2205.15413/) (2022)
- [Simple U-net Based Synthetic Polyp Image Generation: Polyp to Negative and Negative to Polyp](https://mlsanity.com/paper/2302.09835/) (2023)

## Built on by (most cited first, 16 of 16)

- [DiffBoost: Enhancing Medical Image Segmentation via Text-Guided Diffusion Model](https://mlsanity.com/paper/2310.12868/) (2023)
- [MedSegFactory: Text-Guided Generation of Medical Image-Mask Pairs](https://mlsanity.com/paper/2504.06897/) (2025)
- [Polyp-Gen: Realistic and Diverse Polyp Image Generation for Endoscopic Dataset Expansion](https://mlsanity.com/paper/2501.16679/) (2025)
- [Polyp-DDPM: Diffusion-Based Semantic Polyp Synthesis for Enhanced Segmentation](https://mlsanity.com/paper/2402.04031/) (2024)
- [Prompt to Polyp: Medical Text-Conditioned Image Synthesis with Diffusion Models](https://mlsanity.com/paper/2505.05573/) (2025)
- [Diverse Image Generation with Diffusion Models and Cross Class Label Learning for Polyp Classification](https://mlsanity.com/paper/2502.05444/) (2025)
- [Robust Polyp Detection and Diagnosis through Compositional Prompt-Guided Diffusion Models](https://mlsanity.com/paper/2502.17951/) (2025)
- [Multiscale Vector-Quantized Variational Autoencoder for Endoscopic Image Synthesis](https://mlsanity.com/paper/2511.19578/) (2025)
- [What Appears Appealing May Not be Significant! -- A Clinical Perspective of Diffusion Models](https://mlsanity.com/paper/2407.10029/) (2024)
- [LN-Gen: Rectal Lymph Nodes Generation via Anatomical Features](https://mlsanity.com/paper/2408.14977/) (2024)
- [Evaluating Utility of Memory Efficient Medical Image Generation: A Study on Lung Nodule Segmentation](https://mlsanity.com/paper/2410.12542/) (2024)
- [Crucial-Diff: A Unified Diffusion Model for Crucial Image and Annotation Synthesis in Data-scarce Scenarios](https://mlsanity.com/paper/2507.09915/) (2025)
- [Anatomy-Preserving Latent Diffusion for Generation of Brain Segmentation Masks with Ischemic Infarct](https://mlsanity.com/paper/2602.10167/) (2026)
- [Improving Combined Detection and Classification of TEM Defects via Mask-Conditioned Latent Diffusion Augmentation](https://mlsanity.com/paper/2606.02532/) (2026)
- [Representation-driven Endoscopic Visual Embedding Alignment for Latent Generation](https://mlsanity.com/paper/2608.07176/) (2026)
- [Beyond the Foreground: FOV-Aware Polyp Image Synthesis via Lesion-Guided Adaptive Mucosal Context Propagation](https://mlsanity.com/paper/2609.19966/) (2026)

## Abstract (arXiv)

In order to take advantage of AI solutions in endoscopy diagnostics, we must overcome the issue of limited annotations. These limitations are caused by the high privacy concerns in the medical field and the requirement of getting aid from experts for the time-consuming and costly medical data annotation process. In computer vision, image synthesis has made a significant contribution in recent years as a result of the progress of generative adversarial networks (GANs) and diffusion probabilistic models (DPM). Novel DPMs have outperformed GANs in text, image, and video generation tasks. Therefore, this study proposes a conditional DPM framework to generate synthetic GI polyp images conditioned on given generated segmentation masks. Our experimental results show that our system can generate an unlimited number of high-fidelity synthetic polyp images with the corresponding ground truth masks of polyps. To test the usefulness of the generated data, we trained binary image segmentation models to study the effect of using synthetic data. Results show that the best micro-imagewise IOU of 0.7751 was achieved from DeepLabv3+ when the training data consists of both real data and synthetic data. However, the results reflect that achieving good segmentation performance with synthetic data heavily depends on model architectures.