# Membership Inference on Text-to-Image Diffusion Models via Conditional Likelihood Discrepancy

arXiv: https://arxiv.org/abs/2405.14800 · HTML: https://mlsanity.com/paper/2405.14800/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models.
- Builds on 41 of its 64 references that are in the map.
- First posted on arXiv 23 May 2024.

## Placed in

- Diffusion models / [Robustness, privacy and adversarial aspects](https://mlsanity.com/diffusion/robustness-privacy-and-adversarial-aspects/)
- LLM safety and security / [Membership inference attacks](https://mlsanity.com/safety/membership-inference-attacks/)

## Builds on (most cited first, 30 of 41)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [Extracting Training Data from Diffusion Models](https://mlsanity.com/paper/2301.13188/) (2023)
- [DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation](https://mlsanity.com/paper/2110.02711/) (2021)
- [Simple diffusion: End-to-end diffusion for high resolution images](https://mlsanity.com/paper/2301.11093/) (2023)
- [Your Diffusion Model is Secretly a Zero-Shot Classifier](https://mlsanity.com/paper/2303.16203/) (2023)
- [White-box vs Black-box: Bayes Optimal Strategies for Membership Inference](https://mlsanity.com/paper/1908.11229/) (2019)
- [Revisiting Classifier Two-Sample Tests](https://mlsanity.com/paper/1610.06545/) (2016)
- [What the DAAM: Interpreting Stable Diffusion Using Cross Attention](https://mlsanity.com/paper/2210.04885/) (2022)
- [Glaze: Protecting Artists from Style Mimicry by Text-to-Image Models](https://mlsanity.com/paper/2302.04222/) (2023)
- [One Transformer Fits All Distributions in Multi-Modal Diffusion at Scale](https://mlsanity.com/paper/2303.06555/) (2023)
- [Raising the Cost of Malicious AI-Powered Image Editing](https://mlsanity.com/paper/2302.06588/) (2023)
- [A Recipe for Watermarking Diffusion Models](https://mlsanity.com/paper/2303.10137/) (2023)
- [Are Diffusion Models Vulnerable to Membership Inference Attacks?](https://mlsanity.com/paper/2302.01316/) (2023)
- [Navigating Text-To-Image Customization: From LyCORIS Fine-Tuning to Model Evaluation](https://mlsanity.com/paper/2309.14859/) (2023)
- [Text-to-Image Diffusion Models can be Easily Backdoored through Multimodal Data Poisoning](https://mlsanity.com/paper/2305.04175/) (2023)
- [Membership Inference Attacks against Diffusion Models](https://mlsanity.com/paper/2302.03262/) (2023)
- [An empirical study on evaluation metrics of generative adversarial networks](https://mlsanity.com/paper/1806.07755/) (2018)

## Built on by (most cited first, 30 of 38)

- [VBench++: Comprehensive and Versatile Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2411.13503/) (2024)
- [Attacks and Defenses for Generative Diffusion Models: A Comprehensive Survey](https://mlsanity.com/paper/2408.03400/) (2024)
- [CDI: Copyrighted Data Identification in Diffusion Models](https://mlsanity.com/paper/2411.12858/) (2024)
- [Privacy Attacks on Image AutoRegressive Models](https://mlsanity.com/paper/2502.02514/) (2025)
- [Trustworthy Text-to-Image Diffusion Models: A Timely and Focused Survey](https://mlsanity.com/paper/2409.18214/) (2024)
- [ICAS: Detecting Training Data from Autoregressive Image Generative Models](https://mlsanity.com/paper/2507.05068/) (2025)
- [Repairing Catastrophic-Neglect in Text-to-Image Diffusion Models via Attention-Guided Feature Enhancement](https://mlsanity.com/paper/2406.16272/) (2024)
- [Membership Inference Attacks Against Fine-tuned Diffusion Language Models](https://mlsanity.com/paper/2601.20125/) (2026)
- [Responsible Diffusion: A Comprehensive Survey on Safety, Ethics, and Trust in Diffusion Models](https://mlsanity.com/paper/2509.22723/) (2025)
- [Score-based Membership Inference on Diffusion Models](https://mlsanity.com/paper/2509.25003/) (2025)
- [GenAI Confessions: Black-box Membership Inference for Generative Image Models](https://mlsanity.com/paper/2501.06399/) (2025)
- [How does Bayesian Sampling help Membership Inference Attacks?](https://mlsanity.com/paper/2503.07482/) (2025)
- [Auditing Data Provenance in Real-world Text-to-Image Diffusion Models for Privacy and Copyright Protection](https://mlsanity.com/paper/2506.11434/) (2025)
- [DS-Diffusion: Data Style-Guided Diffusion Model for Time-Series Generation](https://mlsanity.com/paper/2509.18584/) (2025)
- [Not Every Time and Frequency Need to Be Forgotten in Diffusion Unlearning](https://mlsanity.com/paper/2510.17917/) (2025)
- [Latent Diffusion Inversion Requires Understanding the Latent Space](https://mlsanity.com/paper/2511.20592/) (2025)
- [Membership and Dataset Inference Attacks on Large Audio Generative Models](https://mlsanity.com/paper/2512.09654/) (2025)
- [Inference Attacks Against Graph Generative Diffusion Models](https://mlsanity.com/paper/2601.03701/) (2026)
- [Identity-Focused Inference and Extraction Attacks on Diffusion Models](https://mlsanity.com/paper/2410.10177/) (2024)
- [Enhancing Membership Inference Attacks on Diffusion Models from a Frequency-Domain Perspective](https://mlsanity.com/paper/2505.20955/) (2025)
- [SoK: Can Synthetic Images Replace Real Data? A Survey of Utility and Privacy of Synthetic Image Generation](https://mlsanity.com/paper/2506.19360/) (2025)
- [AliMark: Enhancing Robustness of Sentence-Level Watermarking Against Text Paraphrasing](https://mlsanity.com/paper/2605.29434/) (2026)
- [Tracing the Roots: Leveraging Temporal Dynamics in Diffusion Trajectories for Origin Attribution](https://mlsanity.com/paper/2411.07449/) (2024)
- [Adversarial Attacks in Multimodal Systems: A Practitioner's Survey](https://mlsanity.com/paper/2505.03084/) (2025)
- [Noise Aggregation Analysis Driven by Small-Noise Injection: Efficient Membership Inference for Diffusion Models](https://mlsanity.com/paper/2510.21783/) (2025)
- [Noise as a Probe: Membership Inference Attacks on Diffusion Models Leveraging Initial Noise](https://mlsanity.com/paper/2601.21628/) (2026)
- [No Caption, No Problem: Caption-Free Membership Inference via Model-Fitted Embeddings](https://mlsanity.com/paper/2602.22689/) (2026)
- [Generalization and Memorization in Rectified Flow](https://mlsanity.com/paper/2603.13421/) (2026)
- [ReproMIA: A Comprehensive Analysis of Model Reprogramming for Proactive Membership Inference Attacks](https://mlsanity.com/paper/2603.28942/) (2026)
- [Black-box Membership Inference Attacks on the Pre-training Data of Image-generation Models](https://mlsanity.com/paper/2605.27020/) (2026)

## Abstract (arXiv)

Text-to-image diffusion models have achieved tremendous success in the field of controllable image generation, while also coming along with issues of privacy leakage and data copyrights. Membership inference arises in these contexts as a potential auditing method for detecting unauthorized data usage. While some efforts have been made on diffusion models, they are not applicable to text-to-image diffusion models due to the high computation overhead and enhanced generalization capabilities. In this paper, we first identify a conditional overfitting phenomenon in text-to-image diffusion models, indicating that these models tend to overfit the conditional distribution of images given the corresponding text rather than the marginal distribution of images only. Based on this observation, we derive an analytical indicator, namely Conditional Likelihood Discrepancy (CLiD), to perform membership inference, which reduces the stochasticity in estimating memorization of individual samples. Experimental results demonstrate that our method significantly outperforms previous methods across various data distributions and dataset scales. Additionally, our method shows superior resistance to overfitting mitigation strategies, such as early stopping and data augmentation.