mlsanityData to 7 Oct 2026

Diffusion Models for Implicit Image Segmentation Ensembles

arXiv 2112.03145 ↗First posted 6 Dec 2021cs.CVcited by 198

On the map: Diffusion models → Medical imaging and segmentation

Abstract (arXiv)

Diffusion models have shown impressive performance for generative modelling of images. In this paper, we present a novel semantic segmentation method based on diffusion models. By modifying the training and sampling scheme, we show that diffusion models can perform lesion segmentation of medical images. To generate an image specific segmentation, we train the model on the ground truth segmentation, and use the image as a prior during training and in every step during the sampling process. With the given stochastic sampling process, we can generate a distribution of segmentation masks. This property allows us to compute pixel-wise uncertainty maps of the segmentation, and allows an implicit ensemble of segmentations that increases the segmentation performance. We evaluate our method on the BRATS2020 dataset for brain tumor segmentation. Compared to state-of-the-art segmentation models, our approach yields good segmentation results and, additionally, detailed uncertainty maps.

Key facts. Cited by 198 papers in the map (data through 7 Oct 2026). Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models. Builds on 26 of its 38 references that are in the map. First posted on arXiv 6 Dec 2021.

Builds on

26 of 38 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021SegDiff: Image Segmentation with Diffusion Probabilistic Models1 Dec

Other references · 25

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2020Denoising Diffusion Implicit Modelscited by 9,655
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  5. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  6. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  7. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  8. 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,483
  9. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
  10. 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 813
  11. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 634
  12. 2018Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challengecited by 508
13 more
  1. 2018A Probabilistic U-Net for Segmentation of Ambiguous Imagescited by 281
  2. 2018Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networkscited by 278
  3. 2015Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understandingcited by 218
  4. 2018Exploring Uncertainty Measures in Deep Networks for Multiple Sclerosis Lesion Detection and Segmentationcited by 146
  5. 2021UNIT-DDPM: UNpaired Image Translation with Denoising Diffusion Probabilistic Modelscited by 133
  6. 2020Stochastic Segmentation Networks: Modelling Spatially Correlated Aleatoric Uncertaintycited by 100
  7. 2019Assessing Reliability and Challenges of Uncertainty Estimations for Medical Image Segmentationcited by 75
  8. 2019A Hierarchical Probabilistic U-Net for Modeling Multi-Scale Ambiguitiescited by 65
  9. 2018Leveraging Uncertainty Estimates for Predicting Segmentation Qualitycited by 44
  10. 2019Automated Design of Deep Learning Methods for Biomedical Image Segmentationcited by 27
  11. 2019On the Validity of Bayesian Neural Networks for Uncertainty Estimationcited by 14
  12. 2020Uncertainty Evaluation Metric for Brain Tumour Segmentationcited by 11
  13. 2020Uncertainty Quantification using Variational Inference for Biomedical Image Segmentationcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 14; 2023: 72; 2024: 61; 2025: 34; 2026: 17.

198 papers.

Sort:

By area:

  1. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  2. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,172
  3. 2022Diffusion Models in Vision: A Surveycited by 1,008
  4. 2023Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentationcited by 485
  5. 2022DiffusionDet: Diffusion Model for Object Detectioncited by 354
  6. 2023A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondencecited by 297
  7. 2022Diffusion Models for Medical Image Analysis: A Comprehensive Surveycited by 269
  8. 2022A Survey on Generative Diffusion Modelcited by 238
  9. 2022MedSegDiff: Medical Image Segmentation with Diffusion Probabilistic Modelcited by 180
  10. 2024Medical SAM 2: Segment medical images as video via Segment Anything Model 2cited by 151
  11. 2023DDP: Diffusion Model for Dense Visual Predictioncited by 148
  12. 2023MedSegDiff-V2: Diffusion based Medical Image Segmentation with Transformercited by 124
38 more
  1. 2023MagicPose: Realistic Human Poses and Facial Expressions Retargeting with Identity-aware Diffusioncited by 115
  2. 2023Dataset Diffusion: Diffusion-based Synthetic Dataset Generation for Pixel-Level Semantic Segmentationcited by 112
  3. 2022A Generalist Framework for Panoptic Segmentation of Images and Videoscited by 99
  4. 2023Ambiguous Medical Image Segmentation using Diffusion Modelscited by 92
  5. 2023DiffusionDepth: Diffusion Denoising Approach for Monocular Depth Estimationcited by 81
  6. 2024Anatomically-Controllable Medical Image Generation with Segmentation-Guided Diffusion Modelscited by 69
  7. 2023Generative AI for Medical Imaging: extending the MONAI Frameworkcited by 68
  8. 2024A Survey on Diffusion Models for Time Series and Spatio-Temporal Datacited by 66
  9. 2024Towards Generalizable Tumor Synthesiscited by 65
  10. 2024WDM: 3D Wavelet Diffusion Models for High-Resolution Medical Image Synthesiscited by 56
  11. 2024A Comprehensive Survey of Foundation Models in Medicinecited by 56
  12. 2022Efficient Diffusion Models for Vision: A Surveycited by 50
  13. 2023FeatureNeRF: Learning Generalizable NeRFs by Distilling Foundation Modelscited by 47
  14. 2023RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Modelcited by 47
  15. 2023Point Cloud Pre-training with Diffusion Modelscited by 47
  16. 2025Self-Correcting Decoding with Generative Feedback for Mitigating Hallucinations in Large Vision-Language Modelscited by 44
  17. 2023Diff-UNet: A Diffusion Embedded Network for Volumetric Segmentationcited by 41
  18. 2023LayerDiffusion: Layered Controlled Image Editing with Diffusion Modelscited by 40
  19. 2023Stochastic Segmentation with Conditional Categorical Diffusion Modelscited by 38
  20. 2023Towards Generic Semi-Supervised Framework for Volumetric Medical Image Segmentationcited by 35
  21. 2024Image Segmentation in Foundation Model Era: A Surveycited by 35
  22. 2023Faster Diffusion: Rethinking the Role of the Encoder for Diffusion Model Inferencecited by 33
  23. 2023DiffMIC: Dual-Guidance Diffusion Network for Medical Image Classificationcited by 32
  24. 2023Memory-Efficient 3D Denoising Diffusion Models for Medical Image Processingcited by 30
  25. 2023Contrastive Conditional Latent Diffusion for Audio-visual Segmentationcited by 30
  26. 2023FreeReg: Image-to-Point Cloud Registration Leveraging Pretrained Diffusion Models and Monocular Depth Estimatorscited by 29
  27. 2023Parallel Vertex Diffusion for Unified Visual Groundingcited by 28
  28. 2024OccGen: Generative Multi-modal 3D Occupancy Prediction for Autonomous Drivingcited by 28
  29. 2023BerDiff: Conditional Bernoulli Diffusion Model for Medical Image Segmentationcited by 27
  30. 20233DiffTection: 3D Object Detection with Geometry-Aware Diffusion Featurescited by 26
  31. 2024Language-driven Grasp Detectioncited by 26
  32. 2024FlowSDF: Flow Matching for Medical Image Segmentation Using Distance Transformscited by 25
  33. 2022The Swiss Army Knife for Image-to-Image Translation: Multi-Task Diffusion Modelscited by 24
  34. 2022A Diffusion Model Predicts 3D Shapes from 2D Microscopy Imagescited by 24
  35. 2023DiffI2I: Efficient Diffusion Model for Image-to-Image Translationcited by 24
  36. 2025USP: Unified Self-Supervised Pretraining for Image Generation and Understandingcited by 24
  37. 2024Around the World in 80 Timesteps: A Generative Approach to Global Visual Geolocationcited by 23
  38. 2024Denoising Diffusion Models for 3D Healthy Brain Tissue Inpaintingcited by 22

50 most cited of 198 All 198 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2021Label-Efficient Semantic Segmentation with Diffusion Models
  2. 2023Robust semi-supervised segmentation with timestep ensembling diffusion models
  3. 2024Latent Diffusion for Medical Image Segmentation: End to end learning for fast sampling and accuracy
  4. 2026Diffusion Model as a Generalist Segmentation Learner
  5. 2024Enhancing Medical Image Segmentation with Deep Learning and Diffusion Models
  6. 2022Fast Unsupervised Brain Anomaly Detection and Segmentation with Diffusion Models
  7. 2023DiffBoost: Enhancing Medical Image Segmentation via Text-Guided Diffusion Model
  8. 2023Masked Diffusion as Self-supervised Representation Learner