# ArSDM: Colonoscopy Images Synthesis with Adaptive Refinement Semantic Diffusion Models

arXiv: https://arxiv.org/abs/2309.01111 · HTML: https://mlsanity.com/paper/2309.01111/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 23 of its 30 references that are in the map.
- First posted on arXiv 3 Sep 2023.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 23 of 23)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [Kvasir-SEG: A Segmented Polyp Dataset](https://mlsanity.com/paper/1911.07069/) (2019)
- [Sparse R-CNN: End-to-End Object Detection with Learnable Proposals](https://mlsanity.com/paper/2011.12450/) (2020)
- [PraNet: Parallel Reverse Attention Network for Polyp Segmentation](https://mlsanity.com/paper/2006.11392/) (2020)
- [Diffusion Models for Medical Anomaly Detection](https://mlsanity.com/paper/2203.04306/) (2022)
- [A Benchmark for Endoluminal Scene Segmentation of Colonoscopy Images](https://mlsanity.com/paper/1612.00799/) (2016)
- [Polyp-PVT: Polyp Segmentation with Pyramid Vision Transformers](https://mlsanity.com/paper/2108.06932/) (2021)
- [Data augmentation using learned transformations for one-shot medical image segmentation](https://mlsanity.com/paper/1902.09383/) (2019)
- [Semantic Image Synthesis via Diffusion Models](https://mlsanity.com/paper/2207.00050/) (2022)
- [Shallow Attention Network for Polyp Segmentation](https://mlsanity.com/paper/2108.00882/) (2021)
- [Fast Unsupervised Brain Anomaly Detection and Segmentation with Diffusion Models](https://mlsanity.com/paper/2206.03461/) (2022)
- [Diffusion Deformable Model for 4D Temporal Medical Image Generation](https://mlsanity.com/paper/2206.13295/) (2022)
- [Adaptive Context Selection for Polyp Segmentation](https://mlsanity.com/paper/2301.04799/) (2023)
- [Lesion-aware Dynamic Kernel for Polyp Segmentation](https://mlsanity.com/paper/2301.04904/) (2023)
- [Semi-supervised Task-driven Data Augmentation for Medical Image Segmentation](https://mlsanity.com/paper/2007.05363/) (2020)
- [Multi-frame Collaboration for Effective Endoscopic Video Polyp Detection via Spatial-Temporal Feature Transformation](https://mlsanity.com/paper/2107.03609/) (2021)

## Built on by (most cited first, 30 of 31)

- [Noise-Consistent Siamese-Diffusion for Medical Image Synthesis and Segmentation](https://mlsanity.com/paper/2505.06068/) (2025)
- [LeFusion: Controllable Pathology Synthesis via Lesion-Focused Diffusion Models](https://mlsanity.com/paper/2403.14066/) (2024)
- [MedSegFactory: Text-Guided Generation of Medical Image-Mask Pairs](https://mlsanity.com/paper/2504.06897/) (2025)
- [Using Diffusion Models to Generate Synthetic Labelled Data for Medical Image Segmentation](https://mlsanity.com/paper/2310.16794/) (2023)
- [Advancing Text-Driven Chest X-Ray Generation with Policy-Based Reinforcement Learning](https://mlsanity.com/paper/2403.06516/) (2024)
- [Masked Conditional Diffusion Model for Enhancing Deepfake Detection](https://mlsanity.com/paper/2402.00541/) (2024)
- [Deformation-Recovery Diffusion Model (DRDM): Instance Deformation for Image Manipulation and Synthesis](https://mlsanity.com/paper/2407.07295/) (2024)
- [MedDiff-FM: A Diffusion-based Foundation Model for Versatile Medical Image Applications](https://mlsanity.com/paper/2410.15432/) (2024)
- [Adaptively Distilled ControlNet: Accelerated Training and Superior Sampling for Medical Image Synthesis](https://mlsanity.com/paper/2507.23652/) (2025)
- [CCIS-Diff: A Generative Model with Stable Diffusion Prior for Controlled Colonoscopy Image Synthesis](https://mlsanity.com/paper/2411.12198/) (2024)
- [Toward Medical Deepfake Detection: A Comprehensive Dataset and Novel Method](https://mlsanity.com/paper/2509.15711/) (2025)
- [CT Synthesis with Conditional Diffusion Models for Abdominal Lymph Node Segmentation](https://mlsanity.com/paper/2403.17770/) (2024)
- [Prompt to Polyp: Medical Text-Conditioned Image Synthesis with Diffusion Models](https://mlsanity.com/paper/2505.05573/) (2025)
- [DeepShade: Enable Shade Simulation by Text-conditioned Image Generation](https://mlsanity.com/paper/2507.12103/) (2025)
- [Synomaly Noise and Multi-Stage Diffusion: A Novel Approach for Unsupervised Anomaly Detection in Medical Images](https://mlsanity.com/paper/2411.04004/) (2024)
- [Diverse Image Generation with Diffusion Models and Cross Class Label Learning for Polyp Classification](https://mlsanity.com/paper/2502.05444/) (2025)
- [Robust Polyp Detection and Diagnosis through Compositional Prompt-Guided Diffusion Models](https://mlsanity.com/paper/2502.17951/) (2025)
- [Towards a Benchmark for Colorectal Cancer Segmentation in Endorectal Ultrasound Videos: Dataset and Model Development](https://mlsanity.com/paper/2408.10067/) (2024)
- [Label-Efficient Data Augmentation with Video Diffusion Models for Guidewire Segmentation in Cardiac Fluoroscopy](https://mlsanity.com/paper/2412.16050/) (2024)
- [PPS-Ctrl: Controllable Sim-to-Real Translation for Colonoscopy Depth Estimation](https://mlsanity.com/paper/2504.17067/) (2025)
- [CLAIM: Clinically-Guided LGE Augmentation for Realistic and Diverse Myocardial Scar Synthesis and Segmentation](https://mlsanity.com/paper/2506.15549/) (2025)
- [AdaptDiff: Cross-Modality Domain Adaptation via Weak Conditional Semantic Diffusion for Retinal Vessel Segmentation](https://mlsanity.com/paper/2410.04648/) (2024)
- [Image Synthesis with Class-Aware Semantic Diffusion Models for Surgical Scene Segmentation](https://mlsanity.com/paper/2410.23962/) (2024)
- [Uncertainty-Aware Regularization for Image-to-Image Translation](https://mlsanity.com/paper/2412.01705/) (2024)
- [Inpainting is All You Need: A Diffusion-based Augmentation Method for Semi-supervised Medical Image Segmentation](https://mlsanity.com/paper/2506.23038/) (2025)
- [Polyp-E: Benchmarking the Robustness of Deep Segmentation Models via Polyp Editing](https://mlsanity.com/paper/2410.16732/) (2024)
- [Breast Ultrasound Tumor Generation via Mask Generator and Text-Guided Network:A Clinically Controllable Framework with Downstream Evaluation](https://mlsanity.com/paper/2507.07721/) (2025)
- [Crucial-Diff: A Unified Diffusion Model for Crucial Image and Annotation Synthesis in Data-scarce Scenarios](https://mlsanity.com/paper/2507.09915/) (2025)
- [LAW & ORDER: Adaptive Spatial Weighting for Medical Diffusion and Segmentation](https://mlsanity.com/paper/2603.04795/) (2026)
- [ShadeBench: A Benchmark Dataset for Building Shade Simulation in Sustainable Society](https://mlsanity.com/paper/2605.20510/) (2026)

## Abstract (arXiv)

Colonoscopy analysis, particularly automatic polyp segmentation and detection, is essential for assisting clinical diagnosis and treatment. However, as medical image annotation is labour- and resource-intensive, the scarcity of annotated data limits the effectiveness and generalization of existing methods. Although recent research has focused on data generation and augmentation to address this issue, the quality of the generated data remains a challenge, which limits the contribution to the performance of subsequent tasks. Inspired by the superiority of diffusion models in fitting data distributions and generating high-quality data, in this paper, we propose an Adaptive Refinement Semantic Diffusion Model (ArSDM) to generate colonoscopy images that benefit the downstream tasks. Specifically, ArSDM utilizes the ground-truth segmentation mask as a prior condition during training and adjusts the diffusion loss for each input according to the polyp/background size ratio. Furthermore, ArSDM incorporates a pre-trained segmentation model to refine the training process by reducing the difference between the ground-truth mask and the prediction mask. Extensive experiments on segmentation and detection tasks demonstrate the generated data by ArSDM could significantly boost the performance of baseline methods.