mlsanityData to 13 Oct 2026

SegDiff: Image Segmentation with Diffusion Probabilistic Models

arXiv 2112.00390 ↗First posted 1 Dec 2021cs.CV cs.AI cs.LGcited by 280

On the map: Diffusion models → Medical imaging and segmentation

Abstract (arXiv)

Diffusion Probabilistic Methods are employed for state-of-the-art image generation. In this work, we present a method for extending such models for performing image segmentation. The method learns end-to-end, without relying on a pre-trained backbone. The information in the input image and in the current estimation of the segmentation map is merged by summing the output of two encoders. Additional encoding layers and a decoder are then used to iteratively refine the segmentation map, using a diffusion model. Since the diffusion model is probabilistic, it is applied multiple times, and the results are merged into a final segmentation map. The new method produces state-of-the-art results on the Cityscapes validation set, the Vaihingen building segmentation benchmark, and the MoNuSeg dataset.

Key facts. Cited by 280 papers in the map (data through 13 Oct 2026). Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models. Builds on 46 of its 54 references that are in the map. First posted on arXiv 1 Dec 2021.

Builds on

46 of 54 references.

References · 46

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,950
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,952
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,656
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,213
  5. 2017Mask R-CNNcited by 7,266
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,023
  7. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,849
  8. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,435
  9. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,751
  10. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,240
  11. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,930
  12. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,685
34 more
  1. 2018UNet++: A Nested U-Net Architecture for Medical Image Segmentationcited by 1,522
  2. 2021Image Super-Resolution via Iterative Refinementcited by 1,509
  3. 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,484
  4. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,348
  5. 2021Palette: Image-to-Image Diffusion Modelscited by 1,298
  6. 2018ESRGAN: Enhanced Super-Resolution Generative Adversarial Networkscited by 1,252
  7. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,159
  8. 2021Variational Diffusion Modelscited by 1,148
  9. 2015Multi-Scale Context Aggregation by Dilated Convolutionscited by 1,129
  10. 2021Segmenter: Transformer for Semantic Segmentationcited by 855
  11. 2018CCNet: Criss-Cross Attention for Semantic Segmentationcited by 718
  12. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 635
  13. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 621
  14. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 446
  15. 2021SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Modelscited by 415
  16. 2020Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentationcited by 326
  17. 2021Medical Transformer: Gated Axial-Attention for Medical Image Segmentationcited by 320
  18. 2017Adversarial Examples for Semantic Segmentation and Object Detectioncited by 275
  19. 2021A Variational Perspective on Diffusion-Based Generative Models and Score Matchingcited by 182
  20. 2021DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanismcited by 154
  21. 2021Rethinking BiSeNet For Real-time Semantic Segmentationcited by 124
  22. 2018Efficient Interactive Annotation of Segmentation Datasets with Polygon-RNN++cited by 120
  23. 2019Fast Interactive Object Annotation with Curve-GCNcited by 95
  24. 2021Noise Estimation for Generative Diffusion Modelscited by 94
  25. 2019PolyTransform: Deep Polygon Transformer for Instance Segmentationcited by 92
  26. 2019HarDNet: A Low Memory Traffic Networkcited by 76
  27. 2016Semantic Segmentation using Adversarial Networkscited by 73
  28. 2020HyperSeg: Patch-wise Hypernetwork for Real-time Semantic Segmentationcited by 66
  29. 2017SegAN: Adversarial Network with Multi-scale L1 Loss for Medical Image Segmentationcited by 49
  30. 2018Learning deep structured active contours end-to-endcited by 38
  31. 2017Adversarial Examples for Semantic Image Segmentationcited by 33
  32. 2019DARNet: Deep Active Ray Network for Building Segmentationcited by 30
  33. 2020End-to-End Trainable Deep Active Contour Models for Automated Image Segmentation: Delineating Buildings in Aerial Imagerycited by 16
  34. 2019End to End Trainable Active Contours via Differentiable Renderingcited by 15

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 1; 2022: 17; 2023: 91; 2024: 74; 2025: 68; 2026: 29.

280 papers.

Sort:

By area:

  1. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,112
  2. 2022Diffusion Models in Vision: A Surveycited by 1,012
  3. 2023Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentationcited by 487
  4. 2023UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Modelscited by 444
  5. 2023A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondencecited by 298
  6. 2022Diffusion Models for Medical Image Analysis: A Comprehensive Surveycited by 273
  7. 2024StoryDiffusion: Consistent Self-Attention for Long-Range Image and Video Generationcited by 241
  8. 2022A Survey on Generative Diffusion Modelcited by 240
  9. 2022Diffusion models as plug-and-play priorscited by 231
  10. 2021Diffusion Models for Implicit Image Segmentation Ensemblescited by 198
  11. 2023Structural Pruning for Diffusion Modelscited by 195
  12. 2022MedSegDiff: Medical Image Segmentation with Diffusion Probabilistic Modelcited by 180
38 more
  1. 2024Medical SAM 2: Segment medical images as video via Segment Anything Model 2cited by 153
  2. 2023DDP: Diffusion Model for Dense Visual Predictioncited by 148
  3. 2023Diffuse, Attend, and Segment: Unsupervised Zero-Shot Segmentation using Stable Diffusioncited by 131
  4. 2023MagicPose: Realistic Human Poses and Facial Expressions Retargeting with Identity-aware Diffusioncited by 117
  5. 2023Collaborative Diffusion for Multi-Modal Face Generation and Editingcited by 116
  6. 2023Dataset Diffusion: Diffusion-based Synthetic Dataset Generation for Pixel-Level Semantic Segmentationcited by 112
  7. 2022A Generalist Framework for Panoptic Segmentation of Images and Videoscited by 99
  8. 2023Ambiguous Medical Image Segmentation using Diffusion Modelscited by 92
  9. 2023DiffusionDepth: Diffusion Denoising Approach for Monocular Depth Estimationcited by 82
  10. 2023ReVersion: Diffusion-Based Relation Inversion from Imagescited by 82
  11. 2022Diffusion Models Beat GANs on Topology Optimizationcited by 79
  12. 2023Diffusion Model is Secretly a Training-free Open Vocabulary Semantic Segmentercited by 76
  13. 2022A Morphology Focused Diffusion Probabilistic Model for Synthesis of Histopathology Imagescited by 74
  14. 2023Diffusion Action Segmentationcited by 74
  15. 2023Image Denoising: The Deep Learning Revolution and Beyond -- A Survey Paper --cited by 71
  16. 2024pix2gestalt: Amodal Segmentation by Synthesizing Wholescited by 70
  17. 2024A Survey on Diffusion Models for Time Series and Spatio-Temporal Datacited by 66
  18. 2023Word-As-Image for Semantic Typographycited by 63
  19. 2023Concept Decomposition for Visual Exploration and Inspirationcited by 61
  20. 2023MomentDiff: Generative Video Moment Retrieval from Random to Realcited by 58
  21. 2023Dif-Fusion: Towards High Color Fidelity in Infrared and Visible Image Fusion with Diffusion Modelscited by 56
  22. 2022Diffusion Adversarial Representation Learning for Self-supervised Vessel Segmentationcited by 55
  23. 2023DiffusionRet: Generative Text-Video Retrieval with Diffusion Modelcited by 52
  24. 2025ConceptAttention: Diffusion Transformers Learn Highly Interpretable Featurescited by 51
  25. 2022DiffusionInst: Diffusion Model for Instance Segmentationcited by 49
  26. 2024Video-Guided Foley Sound Generation with Multimodal Controlscited by 49
  27. 2022JPEG Artifact Correction using Denoising Diffusion Restoration Modelscited by 47
  28. 2023RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Modelcited by 47
  29. 2023Point Cloud Pre-training with Diffusion Modelscited by 47
  30. 2024PhysMotion: Physics-Grounded Dynamics From a Single Imagecited by 45
  31. 2025Self-Correcting Decoding with Generative Feedback for Mitigating Hallucinations in Large Vision-Language Modelscited by 44
  32. 2023TransFusion -- A Transparency-Based Diffusion Model for Anomaly Detectioncited by 43
  33. 2024SatSynth: Augmenting Image-Mask Pairs through Diffusion Models for Aerial Semantic Segmentationcited by 41
  34. 2023Stochastic Segmentation with Conditional Categorical Diffusion Modelscited by 38
  35. 20243D MedDiffusion: A 3D Medical Latent Diffusion Model for Controllable and High-quality Medical Image Generationcited by 37
  36. 2023DiffPose: SpatioTemporal Diffusion Model for Video-Based Human Pose Estimationcited by 36
  37. 2023An Ensemble Score Filter for Tracking High-Dimensional Nonlinear Dynamical Systemscited by 36
  38. 2023Towards Generic Semi-Supervised Framework for Volumetric Medical Image Segmentationcited by 35

50 most cited of 280 All 280 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2021Label-Efficient Semantic Segmentation with Diffusion Models
  2. 2024EmerDiff: Emerging Pixel-level Semantic Knowledge in Diffusion Models
  3. 2024Panoptic Diffusion Models: co-generation of images and segmentation maps
  4. 2023MedSegDiff-V2: Diffusion based Medical Image Segmentation with Transformer
  5. 2023Factorized Diffusion Architectures for Unsupervised Image Generation and Segmentation
  6. 2023Denoising Diffusion Probabilistic Models for Generation of Realistic Fully-Annotated Microscopy Image Data Sets
  7. 2023Diffusion Probabilistic Fields
  8. 2026From Diffusion to Rectified Flow: Rethinking Text-Based Segmentation