mlsanityData to 7 Oct 2026

Membership Inference on Text-to-Image Diffusion Models via Conditional Likelihood Discrepancy

arXiv 2405.14800 ↗First posted 23 May 2024cs.CR cs.CVcited by 38

On the map: Diffusion models → Robustness, privacy and adversarial aspects

also in 1 more
Abstract (arXiv)

Text-to-image diffusion models have achieved tremendous success in the field of controllable image generation, while also coming along with issues of privacy leakage and data copyrights. Membership inference arises in these contexts as a potential auditing method for detecting unauthorized data usage. While some efforts have been made on diffusion models, they are not applicable to text-to-image diffusion models due to the high computation overhead and enhanced generalization capabilities. In this paper, we first identify a conditional overfitting phenomenon in text-to-image diffusion models, indicating that these models tend to overfit the conditional distribution of images given the corresponding text rather than the marginal distribution of images only. Based on this observation, we derive an analytical indicator, namely Conditional Likelihood Discrepancy (CLiD), to perform membership inference, which reduces the stochasticity in estimating memorization of individual samples. Experimental results demonstrate that our method significantly outperforms previous methods across various data distributions and dataset scales. Additionally, our method shows superior resistance to overfitting mitigation strategies, such as early stopping and data augmentation.

Key facts. Cited by 38 papers in the map (data through 7 Oct 2026). Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models. Builds on 41 of its 64 references that are in the map. First posted on arXiv 23 May 2024.

Builds on

41 of 64 references.

Closest ancestors · 12

Same direction, newest first

  1. 2023Black-box Membership Inference Attacks against Fine-tuned Diffusion Models13 Dec
  2. 2023FT-Shield: A Watermark Against Unauthorized Fine-tuning in Text-to-Image Diffusion Models3 Oct
  3. 2023Towards More Realistic Membership Inference Attacks on Large Diffusion Models22 Jun
  4. 2023Unlearnable Examples for Diffusion Models: Protect Data from Unauthorized Exploitation2 Jun
  5. 2023An Efficient Membership Inference Attack for the Diffusion Model by Proximal Initialization26 May
  6. 2023Robust Classification via a Single Diffusion Model24 May
  7. 2023Text-to-Image Diffusion Models can be Easily Backdoored through Multimodal Data Poisoning7 May
  8. 2023A Recipe for Watermarking Diffusion Models17 Mar
  9. 2023Raising the Cost of Malicious AI-Powered Image Editing13 Feb
  10. 2023Membership Inference Attacks against Diffusion Models7 Feb
  11. 2023Are Diffusion Models Vulnerable to Membership Inference Attacks?2 Feb
  12. 2023Extracting Training Data from Diffusion Models30 Jan

Other references · 29

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  7. 2022Classifier-Free Diffusion Guidancecited by 5,717
  8. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  9. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  10. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  11. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  12. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
17 more
  1. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  2. 2021Membership Inference Attacks From First Principlescited by 839
  3. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulationcited by 640
  4. 2023Simple diffusion: End-to-end diffusion for high resolution imagescited by 413
  5. 2023Your Diffusion Model is Secretly a Zero-Shot Classifiercited by 301
  6. 2019White-box vs Black-box: Bayes Optimal Strategies for Membership Inferencecited by 265
  7. 2016Revisiting Classifier Two-Sample Testscited by 258
  8. 2022What the DAAM: Interpreting Stable Diffusion Using Cross Attentioncited by 233
  9. 2023Glaze: Protecting Artists from Style Mimicry by Text-to-Image Modelscited by 200
  10. 2023One Transformer Fits All Distributions in Multi-Modal Diffusion at Scalecited by 185
  11. 2023Navigating Text-To-Image Customization: From LyCORIS Fine-Tuning to Model Evaluationcited by 108
  12. 2018An empirical study on evaluation metrics of generative adversarial networkscited by 81
  13. 2022Membership Inference Attacks Against Text-to-image Generation Modelscited by 66
  14. 2024PromptCharm: Text-to-Image Generation through Multi-modal Prompting and Refinementcited by 61
  15. 2019The Audio Auditor: User-Level Membership Inference in Internet of Things Voice Servicescited by 25
  16. 2018Performing Co-Membership Attacks Against Deep Generative Modelscited by 24
  17. 2024Is My Data in Your AI? Membership Inference Test (MINT) applied to Face Biometricscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 15; 2026: 16.

38 papers.

Sort:

By area:

  1. 2024VBench++: Comprehensive and Versatile Benchmark Suite for Video Generative Modelscited by 42
  2. 2024Attacks and Defenses for Generative Diffusion Models: A Comprehensive Surveycited by 32
  3. 2024CDI: Copyrighted Data Identification in Diffusion Modelscited by 17
  4. 2025Privacy Attacks on Image AutoRegressive Modelscited by 15
  5. 2024Trustworthy Text-to-Image Diffusion Models: A Timely and Focused Surveycited by 6
  6. 2025ICAS: Detecting Training Data from Autoregressive Image Generative Modelscited by 6
  7. 2024Repairing Catastrophic-Neglect in Text-to-Image Diffusion Models via Attention-Guided Feature Enhancementcited by 5
  8. 2026Membership Inference Attacks Against Fine-tuned Diffusion Language Modelscited by 5
  9. 2025Responsible Diffusion: A Comprehensive Survey on Safety, Ethics, and Trust in Diffusion Modelscited by 3
  10. 2025Score-based Membership Inference on Diffusion Modelscited by 3
  11. 2025GenAI Confessions: Black-box Membership Inference for Generative Image Modelscited by 2
  12. 2025How does Bayesian Sampling help Membership Inference Attacks?cited by 2
26 more
  1. 2025Auditing Data Provenance in Real-world Text-to-Image Diffusion Models for Privacy and Copyright Protectioncited by 2
  2. 2025DS-Diffusion: Data Style-Guided Diffusion Model for Time-Series Generationcited by 2
  3. 2025Not Every Time and Frequency Need to Be Forgotten in Diffusion Unlearningcited by 2
  4. 2025Latent Diffusion Inversion Requires Understanding the Latent Spacecited by 2
  5. 2025Membership and Dataset Inference Attacks on Large Audio Generative Modelscited by 2
  6. 2026Inference Attacks Against Graph Generative Diffusion Modelscited by 2
  7. 2024Identity-Focused Inference and Extraction Attacks on Diffusion Modelscited by 1
  8. 2025Enhancing Membership Inference Attacks on Diffusion Models from a Frequency-Domain Perspectivecited by 1
  9. 2025SoK: Can Synthetic Images Replace Real Data? A Survey of Utility and Privacy of Synthetic Image Generationcited by 1
  10. 2024Tracing the Roots: Leveraging Temporal Dynamics in Diffusion Trajectories for Origin Attributioncited by 0
  11. 2025Adversarial Attacks in Multimodal Systems: A Practitioner's Surveycited by 0
  12. 2025Noise Aggregation Analysis Driven by Small-Noise Injection: Efficient Membership Inference for Diffusion Modelscited by 0
  13. 2026Noise as a Probe: Membership Inference Attacks on Diffusion Models Leveraging Initial Noisecited by 0
  14. 2026No Caption, No Problem: Caption-Free Membership Inference via Model-Fitted Embeddingscited by 0
  15. 2026Generalization and Memorization in Rectified Flowcited by 0
  16. 2026ReproMIA: A Comprehensive Analysis of Model Reprogramming for Proactive Membership Inference Attackscited by 0
  17. 2026Black-box Membership Inference Attacks on the Pre-training Data of Image-generation Modelscited by 0
  18. 2026AliMark: Enhancing Robustness of Sentence-Level Watermarking Against Text Paraphrasingcited by 0
  19. 2026MGI: Member vs Generated Inferencecited by 0
  20. 2026Dataset Usage Inference without Shadow Models or Held-out Datacited by 0
  21. 2026One Framework for All: Cross-Modal Membership Inference for Generative Modelscited by 0
  22. 2026Fingerprinting Text-to-Image Diffusion Models via Collapsed Generationcited by 0
  23. 2026IP Protection in the Era of Visual Generative AI: A Surveycited by 0
  24. 2026Membership Inference in Fine-tuned Diffusion Language Models via Token-level Memorization Asymmetrycited by 0
  25. 2026Hearing the Whispers: Black-Box Membership Inference Attacks on Finetuned TTS Modelscited by 0
  26. 2026From Modes to Memories: Characterizing the Scale-Space Dynamics of Diffusion Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Unveiling Structural Memorization: Structural Membership Inference Attack for Text-to-Image Diffusion Models
  2. 2023DIAGNOSIS: Detecting Unauthorized Data Usages in Text-to-image Diffusion Models
  3. 2024Model Will Tell: Training Membership Inference for Diffusion Models
  4. 2023Understanding and Mitigating Copying in Diffusion Models
  5. 2025Finding DoRI: Discovery of Retained Images in Diffusion Models
  6. 2024An Inversion-based Measure of Memorization for Diffusion Models
  7. 2026DIME: Query-Efficient Framework for Membership Inference on Diffusion Models
  8. 2024Unveiling and Mitigating Memorization in Text-to-image Diffusion Models through Cross Attention