mlsanityData to 7 Oct 2026

Extracting Training Data from Diffusion Models

arXiv 2301.13188 ↗First posted 30 Jan 2023cs.CR cs.CV cs.LGcited by 723

On the map: Diffusion models → Robustness, privacy and adversarial aspects

Abstract (arXiv)

Image diffusion models such as DALL-E 2, Imagen, and Stable Diffusion have attracted significant attention due to their ability to generate high-quality synthetic images. In this work, we show that diffusion models memorize individual images from their training data and emit them at generation time. With a generate-and-filter pipeline, we extract over a thousand training examples from state-of-the-art models, ranging from photographs of individual people to trademarked company logos. We also train hundreds of diffusion models in various settings to analyze how different modeling and data decisions affect privacy. Overall, our results show that diffusion models are much less private than prior generative models such as GANs, and that mitigating these vulnerabilities may require new advances in privacy-preserving training.

Key facts. Cited by 723 papers in the map (data through 7 Oct 2026). Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models. Builds on 60 of its 78 references that are in the map. First posted on arXiv 30 Jan 2023.

Builds on

60 of 78 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Membership Inference of Diffusion Models24 Jan
  2. 2022Differentially Private Diffusion Models18 Oct

Other references · 58

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  8. 2022Generative Adversarial Networkscited by 5,603
  9. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  10. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  11. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  12. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
38 more
  1. 2016Improved Techniques for Training GANscited by 2,938
  2. 2016Deep Learning with Differential Privacycited by 2,682
  3. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  4. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  5. 2020Extracting Training Data from Large Language Modelscited by 2,144
  6. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  7. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
  8. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  9. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  10. 2021Membership Inference Attacks From First Principlescited by 839
  11. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  12. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  13. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 492
  14. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 452
  15. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 452
  16. 2022Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Modelscited by 418
  17. 2020Differentiable Augmentation for Data-Efficient GAN Trainingcited by 361
  18. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 309
  19. 2022Brain Imaging Generation with Latent Diffusion Modelscited by 267
  20. 2019The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networkscited by 241
  21. 2021Enhanced Membership Inference Attacks against Machine Learning Modelscited by 237
  22. 2020Auditing Differentially Private Machine Learning: How Private is Private SGD?cited by 213
  23. 2022What Does it Mean for a Language Model to Preserve Privacy?cited by 194
  24. 2022Reconstructing Training Data with Informed Adversariescited by 143
  25. 2016Mode Regularized Generative Adversarial Networkscited by 123
  26. 2020Revisiting Membership Inference Under Realistic Assumptionscited by 108
  27. 2021Generative Models as a Data Source for Multiview Representation Learningcited by 104
  28. 2022RoentGen: Vision-Language Foundation Model for Chest X-ray Generationcited by 89
  29. 2017Ethical Challenges in Data-Driven Dialogue Systemscited by 86
  30. 2022Position: Considerations for Differentially Private Learning with Large-Scale Public Pretrainingcited by 86
  31. 2022Adapting Pretrained Vision-Language Foundational Models to Medical Imaging Domainscited by 85
  32. 2018GANs for Medical Image Analysiscited by 81
  33. 2022Membership Inference Attacks Against Text-to-image Generation Modelscited by 66
  34. 2021On Memorization in Probabilistic Deep Generative Modelscited by 60
  35. 2022Are Attribute Inference Attacks Just Imputation?cited by 48
  36. 2021Negative Data Augmentationcited by 46
  37. 2018Metropolis-Hastings Generative Adversarial Networkscited by 40
  38. 2022StudioGAN: A Taxonomy and Benchmark of GANs for Image Synthesiscited by 38

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 172; 2024: 208; 2025: 200; 2026: 143.

723 papers.

Sort:

By area:

  1. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  2. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  3. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  4. 2023DIRE for Diffusion-Generated Image Detectioncited by 386
  5. 2023Unified Concept Editing in Diffusion Modelscited by 377
  6. 2023Ablating Concepts in Text-to-Image Diffusion Modelscited by 312
  7. 2023Tool Learning with Foundation Modelscited by 302
  8. 2024MACE: Mass Concept Erasure in Diffusion Modelscited by 298
  9. 2023Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Modelscited by 297
  10. 2023Effective Data Augmentation With Diffusion Modelscited by 282
  11. 2023A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPTcited by 221
  12. 2023Understanding and Mitigating Copying in Diffusion Modelscited by 205
38 more
  1. 2023Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion modelscited by 199
  2. 2024Stable Audio Opencited by 192
  3. 2023Generalization in diffusion models arises from geometry-adaptive harmonic representationscited by 179
  4. 2023SneakyPrompt: Jailbreaking Text-to-image Generative Modelscited by 175
  5. 2023A Categorical Archive of ChatGPT Failurescited by 167
  6. 2023One-Dimensional Adapter to Rule Them All: Concepts, Diffusion Models and Erasing Applicationscited by 164
  7. 2023Holistic Evaluation of Text-To-Image Modelscited by 160
  8. 2023Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarkscited by 139
  9. 2024Benchmark Data Contamination of Large Language Models: A Surveycited by 138
  10. 2023Are Diffusion Models Vulnerable to Membership Inference Attacks?cited by 135
  11. 2023Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4cited by 123
  12. 2023On Memorization in Diffusion Modelscited by 122
  13. 2023Machine Learning for Synthetic Data Generation: A Reviewcited by 118
  14. 2023Receler: Reliable Concept Erasing of Text-to-Image Diffusion Models via Lightweight Eraserscited by 118
  15. 2023Foundation Models and Fair Usecited by 112
  16. 2023Regulating ChatGPT and other Large Generative AI Modelscited by 102
  17. 2024Min-K%++: Improved Baseline for Detecting Pre-Training Data from Large Language Modelscited by 102
  18. 2023On Provable Copyright Protection for Generative Modelscited by 97
  19. 2023MusicLDM: Enhancing Novelty in Text-to-Music Generation Using Beat-Synchronous Mixup Strategiescited by 94
  20. 2024Detecting, Explaining, and Mitigating Memorization in Diffusion Modelscited by 94
  21. 2023REaLTabFormer: Generating Realistic Relational and Tabular Data using Transformerscited by 93
  22. 2023Art and the science of generative AI: A deeper divecited by 93
  23. 2023Leaving Reality to Imagination: Robust Classification via Generated Datasetscited by 91
  24. 2023Evaluating the Social Impact of Generative AI Systems in Systems and Societycited by 91
  25. 2023Generative models improve fairness of medical classifiers under distribution shiftscited by 90
  26. 2024A Phase Transition in Diffusion Models Reveals the Hierarchical Nature of Datacited by 88
  27. 2024Long-form music generation with latent diffusioncited by 88
  28. 2024Understanding Hallucinations in Diffusion Models through Mode Interpolationcited by 87
  29. 2023Differentially Private Diffusion Models Generate Useful Synthetic Imagescited by 85
  30. 2023Membership Inference Attacks against Diffusion Modelscited by 84
  31. 2023When Foundation Model Meets Federated Learning: Motivations, Challenges, and Future Directionscited by 83
  32. 2023Circumventing Concept Erasure Methods For Text-to-Image Generative Modelscited by 81
  33. 2025Why Diffusion Models Don't Memorize: The Role of Implicit Dynamical Regularization in Trainingcited by 78
  34. 2023Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilitiescited by 76
  35. 2024Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systemscited by 76
  36. 2024No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performancecited by 75
  37. 2024Diffusion Models in Low-Level Vision: A Surveycited by 73
  38. 2023The Bias Amplification Paradox in Text-to-Image Generationcited by 69

50 most cited of 723 All 723 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Model Will Tell: Training Membership Inference for Diffusion Models
  2. 2024Learning Differentially Private Diffusion Models via Stochastic Adversarial Distillation
  3. 2025Latent Diffusion Unlearning: Protecting Against Unauthorized Personalization Through Trajectory Shifted Perturbations
  4. 2025From Easy to Hard: Building a Shortcut for Differentially Private Image Synthesis
  5. 2023Unlearnable Examples for Diffusion Models: Protect Data from Unauthorized Exploitation
  6. 2024Gradient-Guided Conditional Diffusion Models for Private Image Reconstruction: Analyzing Adversarial Impacts of Differential Privacy and Denoising
  7. 2024Leveraging Programmatically Generated Synthetic Data for Differentially Private Diffusion Training
  8. 2023DiffProtect: Generate Adversarial Examples with Diffusion Models for Facial Privacy Protection