mlsanityData to 14 Oct 2026

Beware of diffusion models for synthesizing medical images -- A comparison with GANs in terms of memorizing brain MRI and chest x-ray images

arXiv 2305.07644 ↗First posted 12 May 2023eess.IV cs.CV cs.LGcited by 23

On the map: Diffusion models → Medical imaging and segmentation

Abstract (arXiv)

Diffusion models were initially developed for text-to-image generation and are now being utilized to generate high quality synthetic images. Preceded by GANs, diffusion models have shown impressive results using various evaluation metrics. However, commonly used metrics such as FID and IS are not suitable for determining whether diffusion models are simply reproducing the training images. Here we train StyleGAN and a diffusion model, using BRATS20, BRATS21 and a chest x-ray pneumonia dataset, to synthesize brain MRI and chest x-ray images, and measure the correlation between the synthetic images and all training images. Our results show that diffusion models are more likely to memorize the training images, compared to StyleGAN, especially for small datasets and when using 2D slices from 3D volumes. Researchers should be careful when using diffusion models (and to some extent GANs) for medical imaging, if the final goal is to share the synthetic images.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models. Builds on 26 of its 41 references that are in the map. First posted on arXiv 12 May 2023.

Builds on

26 of 41 references.

Closest ancestors · 5

Same direction, newest first

  1. 2022Diffusion Models for Medical Image Analysis: A Comprehensive Survey14 Nov
  2. 2022Generation of Anonymous Chest Radiographs Using Latent Diffusion Models for Training Thoracic Abnormality Classification Systems2 Nov
  3. 2022Spot the fake lungs: Generating Synthetic Medical Images using Neural Diffusion Models2 Nov
  4. 2022A Morphology Focused Diffusion Probabilistic Model for Synthesis of Histopathology Images27 Sep
  5. 2022Brain Imaging Generation with Latent Diffusion Models15 Sep

Other references · 21

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  6. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  7. 2022Generative Adversarial Networkscited by 5,627
  8. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  9. 2016Improved Techniques for Training GANscited by 2,970
  10. 2016Deep Learning with Differential Privacycited by 2,728
  11. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  12. 2023Extracting Training Data from Diffusion Modelscited by 732
9 more
  1. 2018Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challengecited by 511
  2. 2022Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Modelscited by 424
  3. 2018Generative Adversarial Network in Medical Imaging: A Reviewcited by 257
  4. 2018Pros and Cons of GAN Evaluation Measurescited by 224
  5. 2021Pros and Cons of GAN Evaluation Measures: New Developmentscited by 123
  6. 2021Deep Learning for Chest X-ray Analysis: A Surveycited by 84
  7. 2021Ensembles of GANs for synthetic training data generationcited by 6
  8. 2022On the Interpretability of Regularisation for Neural Networks Through Model Gradient Similaritycited by 4
  9. 2022Does an ensemble of GANs lead to better performance when training segmentation networks with synthetic images?cited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 5; 2025: 11; 2026: 4.

23 papers.

Sort:

By area:

  1. 2024A Geometric Framework for Understanding Memorization in Generative Modelscited by 40
  2. 2023Brain tumor segmentation using synthetic MR images -- A comparison of GANs and diffusion modelscited by 19
  3. 2025Metrics that matter: Evaluating image quality metrics for medical image generationcited by 12
  4. 2024Replication in Visual Diffusion Models: A Survey and Outlookcited by 11
  5. 2023Assessing the capacity of a denoising diffusion probabilistic model to reproduce spatial contextcited by 7
  6. 2023Boosting Data Analytics With Synthetic Volume Expansioncited by 7
  7. 2024U Can't Gen This? A Survey of Intellectual Property Protection Methods for Data in Generative AIcited by 7
  8. 2025CheXGenBench: A Unified Benchmark For Fidelity, Privacy and Utility of Synthetic Chest Radiographscited by 5
  9. 2024MemControl: Mitigating Memorization in Diffusion Models via Automated Parameter Selectioncited by 4
  10. 2025Benchmarking GANs, Diffusion Models, and Flow Matching for T1w-to-T2w MRI Translationcited by 4
  11. 2025The Devil is in the Prompts: De-Identification Traces Enhance Memorization Risks in Synthetic Chest X-Ray Generationcited by 3
  12. 2025Frequency-Calibrated Membership Inference Attacks on Medical Image Diffusion Modelscited by 3
11 more
  1. 2025Generalist Models in Medical Image Segmentation: A Survey and Performance Comparison with Task-Specific Approachescited by 2
  2. 2025Can Diffusion Models Bridge the Domain Gap in Cardiac MR Imaging?cited by 2
  3. 2025Transport Novelty Distance: A Distributional Metric for Evaluating Material Generative Modelscited by 2
  4. 2026TumorFlow: Physics-Guided Longitudinal MRI Synthesis of Glioblastoma Growthcited by 2
  5. 2025Beyond the Generative Learning Trilemma: Generative Model Assessment in Data Scarcity Domainscited by 1
  6. 2026Cross Modality Image Translation In Medical Imaging Using Generative Frameworkscited by 1
  7. 2024Generative artificial intelligence in ophthalmology: multimodal retinal images for the diagnosis of Alzheimer's disease with convolutional neural networkscited by 0
  8. 2025Learning and Generating Diverse Residential Load Patterns Using GAN with Weakly-Supervised Training and Weight Selectioncited by 0
  9. 2025SkinGenBench: Generative Model and Preprocessing Effects for Synthetic Dermoscopic Augmentation in Melanoma Diagnosiscited by 0
  10. 2026A Calibrated Memorization Index (MI) for Detecting Training Data Leakage in Generative MRI Modelscited by 0
  11. 2026HUydra: Full-Range Lung CT Synthesis via Multiple HU Interval Generative Modellingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Perceptual Evaluation of GANs and Diffusion Models for Generating X-rays
  2. 2024Diffusion-Based Approaches in Medical Image Generation and Analysis
  3. 2023Evaluating the feasibility of using Generative Models to generate Chest X-Ray Data
  4. 2026DiffusionXRay: A Diffusion and GAN-Based Approach for Enhancing Digitally Reconstructed Chest Radiographs
  5. 2024Cross-conditioned Diffusion Model for Medical Image to Image Translation
  6. 2022Medical Diffusion: Denoising Diffusion Probabilistic Models for 3D Medical Image Generation
  7. 2024DiNO-Diffusion. Scaling Medical Diffusion via Self-Supervised Pre-Training
  8. 20262D Versus 3D Diffusion for In Silico Training of Interventional X-ray AI Models