# Stable Diffusion Segmentation for Biomedical Images with Single-step Reverse Process

arXiv: https://arxiv.org/abs/2406.18361 · HTML: https://mlsanity.com/paper/2406.18361/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 21 of its 27 references that are in the map.
- First posted on arXiv 26 Jun 2024.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 21 of 21)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [Kvasir-SEG: A Segmented Polyp Dataset](https://mlsanity.com/paper/1911.07069/) (2019)
- [Label-Efficient Semantic Segmentation with Diffusion Models](https://mlsanity.com/paper/2112.03126/) (2021)
- [Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysis](https://mlsanity.com/paper/2111.14791/) (2021)
- [REFUGE Challenge: A Unified Framework for Evaluating Automated Methods for Glaucoma Assessment from Fundus Photographs](https://mlsanity.com/paper/1910.03667/) (2019)
- [MedSegDiff: Medical Image Segmentation with Diffusion Probabilistic Model](https://mlsanity.com/paper/2211.00611/) (2022)
- [MedSegDiff-V2: Diffusion based Medical Image Segmentation with Transformer](https://mlsanity.com/paper/2301.11798/) (2023)
- [Stepwise Feature Fusion: Local Guides Global](https://mlsanity.com/paper/2203.03635/) (2022)
- [Diff-UNet: A Diffusion Embedded Network for Volumetric Segmentation](https://mlsanity.com/paper/2303.10326/) (2023)
- [CTooth: A Fully Annotated 3D Dataset and Benchmark for Tooth Volume Segmentation on Cone Beam Computed Tomography Images](https://mlsanity.com/paper/2206.08778/) (2022)
- [CTooth+: A Large-scale Dental Cone Beam Computed Tomography Dataset and Benchmark for Tooth Volume Segmentation](https://mlsanity.com/paper/2208.01643/) (2022)

## Built on by (most cited first, 16 of 16)

- [Are Pixel-Wise Metrics Reliable for Sparse-View Computed Tomography Reconstruction?](https://mlsanity.com/paper/2506.02093/) (2025)
- [PGP-SAM: Prototype-Guided Prompt Learning for Efficient Few-Shot Medical Image Segmentation](https://mlsanity.com/paper/2501.06692/) (2025)
- [LDPoly: Latent Diffusion for Polygonal Road Outline Extraction in Large-Scale Topographic Mapping](https://mlsanity.com/paper/2504.20645/) (2025)
- [Diffusion-Based Data Augmentation for Medical Image Segmentation](https://mlsanity.com/paper/2508.17844/) (2025)
- [ACPV-Net: All-Class Polygonal Vectorization for Seamless Vector Map Generation from Aerial Imagery](https://mlsanity.com/paper/2603.16616/) (2026)
- [SoK: Security and Privacy Risks of Healthcare AI](https://mlsanity.com/paper/2409.07415/) (2024)
- [UniSegDiff: Boosting Unified Lesion Segmentation via a Staged Diffusion Model](https://mlsanity.com/paper/2507.18362/) (2025)
- [Taming Stable Diffusion for Computed Tomography Blind Super-Resolution](https://mlsanity.com/paper/2506.11496/) (2025)
- [LEAF: Latent Diffusion with Efficient Encoder Distillation for Aligned Features in Medical Image Segmentation](https://mlsanity.com/paper/2507.18214/) (2025)
- [Wave-GMS: Lightweight Multi-Scale Generative Model for Medical Image Segmentation](https://mlsanity.com/paper/2510.03216/) (2025)
- [ProGiDiff: Prompt-Guided Diffusion-Based Medical Image Segmentation](https://mlsanity.com/paper/2601.16060/) (2026)
- [DDS-UDA: Dual-Domain Synergy for Unsupervised Domain Adaptation in Joint Segmentation of Optic Disc and Optic Cup](https://mlsanity.com/paper/2603.13345/) (2026)
- [SD-FSMIS: Adapting Stable Diffusion for Few-Shot Medical Image Segmentation](https://mlsanity.com/paper/2604.03134/) (2026)
- [GET: Generative Embedding Translation for Medical Image Segmentation](https://mlsanity.com/paper/2608.22619/) (2026)
- [InstEditSeg: Instruction-Driven Image Editing for Polyp and Skin Lesion Segmentation](https://mlsanity.com/paper/2609.02004/) (2026)
- [Rethinking Diffusion Segmentation: When Does It Rely on Its Noisy State, and Does Diffusion Matter?](https://mlsanity.com/paper/2609.23967/) (2026)

## Abstract (arXiv)

Diffusion models have demonstrated their effectiveness across various generative tasks. However, when applied to medical image segmentation, these models encounter several challenges, including significant resource and time requirements. They also necessitate a multi-step reverse process and multiple samples to produce reliable predictions. To address these challenges, we introduce the first latent diffusion segmentation model, named SDSeg, built upon stable diffusion (SD). SDSeg incorporates a straightforward latent estimation strategy to facilitate a single-step reverse process and utilizes latent fusion concatenation to remove the necessity for multiple samples. Extensive experiments indicate that SDSeg surpasses existing state-of-the-art methods on five benchmark datasets featuring diverse imaging modalities. Remarkably, SDSeg is capable of generating stable predictions with a solitary reverse step and sample, epitomizing the model's stability as implied by its name. The code is available at https://github.com/lin-tianyu/Stable-Diffusion-Seg