mlsanityData to 7 Oct 2026

Anatomically-Controllable Medical Image Generation with Segmentation-Guided Diffusion Models

arXiv 2402.05210 ↗First posted 7 Feb 2024eess.IV cs.CV cs.LGcited by 69

On the map: Diffusion models → Medical imaging and segmentation

Abstract (arXiv)

Diffusion models have enabled remarkably high-quality medical image generation, yet it is challenging to enforce anatomical constraints in generated images. To this end, we propose a diffusion model-based method that supports anatomically-controllable medical image generation, by following a multi-class anatomical segmentation mask at each sampling step. We additionally introduce a random mask ablation training algorithm to enable conditioning on a selected combination of anatomical constraints while allowing flexibility in other anatomical areas. We compare our method ("SegGuidedDiff") to existing methods on breast MRI and abdominal/neck-to-pelvis CT datasets with a wide range of anatomical objects. Results show that our method reaches a new state-of-the-art in the faithfulness of generated images to input anatomical masks on both datasets, and is on par for general anatomical realism. Finally, our model also enjoys the extra benefit of being able to adjust the anatomical similarity of generated images to real images of choice through interpolation in its latent space. SegGuidedDiff has many applications, including cross-modality translation, and the generation of paired or counterfactual data. Our code is available at https://github.com/mazurowski-lab/segmentation-guided-diffusion.

Key facts. Cited by 69 papers in the map (data through 7 Oct 2026). Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models. Builds on 29 of its 40 references that are in the map. First posted on arXiv 7 Feb 2024.

Builds on

29 of 40 references.

Closest ancestors · 6

Same direction, newest first

  1. 2022Diffusion Models for Medical Image Analysis: A Comprehensive Survey14 Nov
  2. 2022Brain Imaging Generation with Latent Diffusion Models15 Sep
  3. 2022PET image denoising based on denoising diffusion probabilistic models13 Sep
  4. 2022Fast Unsupervised Brain Anomaly Detection and Segmentation with Diffusion Models7 Jun
  5. 2022Diffusion Models for Medical Anomaly Detection8 Mar
  6. 2021Diffusion Models for Implicit Image Segmentation Ensembles6 Dec

Other references · 23

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  8. 2022Classifier-Free Diffusion Guidancecited by 5,717
  9. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  10. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  11. 2013Auto-Encoding Variational Bayescited by 3,980
  12. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
11 more
  1. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,295
  2. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  3. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
  4. 2017StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translationcited by 1,036
  5. 2022Pretraining is All You Need for Image-to-Image Translationcited by 160
  6. 2022Conversion Between CT and MRI Images Using Diffusion and Score-Matching Modelscited by 69
  7. 2019Unsupervised Domain Adaptation via Disentangled Representations: Application to Cross-Modality Liver Segmentationcited by 50
  8. 2022Can segmentation models be trained with fully synthetically generated data?cited by 37
  9. 2023Reverse Engineering Breast MRIs: Predicting Acquisition Parameters Directly from Imagescited by 7
  10. 2023Deep Learning for Breast MRI Style Transfer with Limited Training Datacited by 6
  11. 2023A 3D generative model of pathological multi-modal MR images and segmentationscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 16; 2025: 30; 2026: 23.

69 papers.

Sort:

By area:

  1. 2024MAISI: Medical AI for Synthetic Imagingcited by 86
  2. 2024Fr'echet Radiomic Distance (FRD): A Versatile Metric for Comparing Medical Imaging Datasetscited by 19
  3. 2024vesselFM: A Foundation Model for Universal 3D Blood Vessel Segmentationcited by 18
  4. 2024Deep Generative Models for 3D Medical Image Synthesiscited by 14
  5. 2024Towards Learning Contrast Kinetics with Multi-Condition Latent Diffusion Modelscited by 11
  6. 2024XReal: Realistic Anatomy and Pathology-Aware X-ray Generation via Controllable Diffusion Modelcited by 8
  7. 2025PathDiff: Histopathology Image Synthesis with Unpaired Text and Mask Conditionscited by 8
  8. 2025TRACE: Temporally Reliable Anatomically-Conditioned 3D CT Generation with Enhanced Efficiencycited by 8
  9. 2025Generative Artificial Intelligence in Medical Imaging: Foundations, Progress, and Clinical Translationcited by 8
  10. 2024Simulating Dynamic Tumor Contrast Enhancement in Breast MRI using Conditional Generative Adversarial Networkscited by 6
  11. 2025Toward Medical Deepfake Detection: A Comprehensive Dataset and Novel Methodcited by 6
  12. 2024Physics-Inspired Generative Models in Medical Imaging: A Reviewcited by 5
38 more
  1. 2024Generalizable Single-Source Cross-modality Medical Image Segmentation via Invariant Causal Mechanismscited by 5
  2. 2025No More Sliding Window: Efficient 3D Medical Image Segmentation with Differentiable Top-k Patch Samplingcited by 5
  3. 2024Rethinking Perceptual Metrics for Medical Image Translationcited by 4
  4. 2025BrainMRDiff: A Diffusion Model for Anatomically Consistent Brain MRI Synthesiscited by 4
  5. 2024MRGen: Segmentation Data Engine for Underrepresented MRI Modalitiescited by 3
  6. 2025Med-Art: Diffusion Transformer for 2D Medical Text-to-Image Generationcited by 3
  7. 2025Diffusion-Based User-Guided Data Augmentation for Coronary Stenosis Detectioncited by 3
  8. 2025Comparing Conditional Diffusion Models for Synthesizing Contrast-Enhanced Breast MRI from Pre-Contrast Imagescited by 3
  9. 2025Mechanistic Learning with Guided Diffusion Models to Predict Spatio-Temporal Brain Tumor Growthcited by 3
  10. 2025Semantic and Visual Crop-Guided Diffusion Models for Heterogeneous Tissue Synthesis in Histopathologycited by 3
  11. 2024ContourDiff: Unpaired Medical Image Translation with Structural Consistencycited by 2
  12. 2024Label-Efficient Data Augmentation with Video Diffusion Models for Guidewire Segmentation in Cardiac Fluoroscopycited by 2
  13. 2025LesiOnTime -- Joint Temporal and Clinical Modeling for Small Breast Lesion Segmentation in Longitudinal DCE-MRIcited by 2
  14. 2025Latent Interpolation Learning Using Diffusion Models for Cardiac Volume Reconstructioncited by 2
  15. 2025RainDiff: End-to-end Precipitation Nowcasting Via Token-wise Attention Diffusioncited by 2
  16. 2025Diffusion-Driven Generation of Minimally Preprocessed Brain MRIcited by 2
  17. 2024Diffusion-based Data Augmentation and Knowledge Distillation with Generated Soft Labels Solving Data Scarcity Problems of SAR Oil Spill Segmentationcited by 1
  18. 2025Physics-informed DeepCT: Sinogram Wavelet Decomposition Meets Masked Diffusioncited by 1
  19. 2025Interactive Tumor Progression Modeling via Sketch-Based Image Editingcited by 1
  20. 2025Schr"odinger Diffusion Driven Signal Recovery in 3T BOLD fMRI Using Unmatched 7T Observationscited by 1
  21. 2025Shifts in Doctors' Eye Movements Between Real and AI-Generated Medical Imagescited by 1
  22. 2025ViCTr: Vital Consistency Transfer for Pathology Aware Image Synthesiscited by 1
  23. 2025Paired Image Generation with Diffusion-Guided Diffusion Modelscited by 1
  24. 2025Joint Holistic and Lesion Controllable Mammogram Synthesis via Gated Conditional Diffusion Modelcited by 1
  25. 2025Diffusion-Guided Mask-Consistent Paired Mixing for Endoscopic Image Segmentationcited by 1
  26. 2026Cortex-Grounded Diffusion Models for Brain Image Generationcited by 1
  27. 2026Geometrically Constrained Stenosis Editing in Coronary Angiography via Entropic Optimal Transportcited by 1
  28. 2026Treatment-Conditioned Diffusion for Forecasting Neurodegenerative Disease Progressioncited by 1
  29. 2026Improving Medical Image Generative Models with Fr'echet Distance Losscited by 1
  30. 2024GuideGen: A Text-Guided Framework for Paired Full-torso Anatomy and CT Volume Generationcited by 0
  31. 2024Evaluating Utility of Memory Efficient Medical Image Generation: A Study on Lung Nodule Segmentationcited by 0
  32. 2025Diff-Lung: Diffusion-Based Texture Synthesis for Enhanced Pathological Tissue Segmentation in Lung CT Scanscited by 0
  33. 2025ZECO: ZeroFusion Guided 3D MRI Conditional Generationcited by 0
  34. 2025Phase-fraction guided denoising diffusion model for augmenting multiphase steel microstructure segmentation via micrograph image-mask pair synthesiscited by 0
  35. 2025AnatoMaskGAN: GNN-Driven Slice Feature Fusion and Noise Augmentation for Medical Semantic Image Synthesiscited by 0
  36. 2025DEMIST: Decoupled Multi-stream latent diffusion for Quantitative Myelin Map Synthesiscited by 0
  37. 2025BrainNormalizer: Anatomy-Informed Pseudo-Healthy Brain Reconstruction from Tumor MRI via Edge-Guided ControlNetcited by 0
  38. 2025PathCo-LatticE: Pathology-Constrained Lattice-Of Experts Framework for Fully-supervised Few-Shot Cardiac MRI Segmentationcited by 0

50 most cited of 69 All 69 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025MedDiff-FT: Data-Efficient Diffusion Model Fine-tuning with Structural Guidance for Controllable Medical Image Synthesis
  2. 2025seg2med: a bridge from artificial anatomy to multimodal medical images
  3. 2023DiffBoost: Enhancing Medical Image Segmentation via Text-Guided Diffusion Model
  4. 2024MedDiff-FM: A Diffusion-based Foundation Model for Versatile Medical Image Applications
  5. 2024Data Augmentation for Surgical Scene Segmentation with Anatomy-Aware Diffusion Models
  6. 2025Noise-Consistent Siamese-Diffusion for Medical Image Synthesis and Segmentation
  7. 2022Medical Diffusion: Denoising Diffusion Probabilistic Models for 3D Medical Image Generation
  8. 2026ProGiDiff: Prompt-Guided Diffusion-Based Medical Image Segmentation