# Membership Inference Attacks Against Text-to-image Generation Models

arXiv: https://arxiv.org/abs/2210.00968 · HTML: https://mlsanity.com/paper/2210.00968/

## Key facts

- Cited by 69 papers in the map (data through 14 Oct 2026).
- Placed in Membership inference attacks, one of 12 directions in LLM safety and security.
- Builds on 33 of its 39 references that are in the map.
- First posted on arXiv 3 Oct 2022.

## Placed in

- LLM safety and security / [Membership inference attacks](https://mlsanity.com/safety/membership-inference-attacks/)

## Builds on (most cited first, 30 of 33)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [Microsoft COCO Captions: Data Collection and Evaluation Server](https://mlsanity.com/paper/1504.00325/) (2015)
- [Scaling Autoregressive Models for Content-Rich Text-to-Image Generation](https://mlsanity.com/paper/2206.10789/) (2022)
- [Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts](https://mlsanity.com/paper/2102.08981/) (2021)
- [CogView: Mastering Text-to-Image Generation via Transformers](https://mlsanity.com/paper/2105.13290/) (2021)
- [ClipCap: CLIP Prefix for Image Captioning](https://mlsanity.com/paper/2111.09734/) (2021)
- [ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models](https://mlsanity.com/paper/1806.01246/) (2018)
- [Label-Only Membership Inference Attacks](https://mlsanity.com/paper/2007.14321/) (2020)
- [Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooring](https://mlsanity.com/paper/1802.04633/) (2018)
- [Embedding Watermarks into Deep Neural Networks](https://mlsanity.com/paper/1701.04082/) (2017)
- [General Facial Representation Learning in a Visual-Linguistic Manner](https://mlsanity.com/paper/2112.03109/) (2021)
- [Membership Leakage in Label-Only Exposures](https://mlsanity.com/paper/2007.15528/) (2020)
- [RetinaFace: Single-stage Dense Face Localisation in the Wild](https://mlsanity.com/paper/1905.00641/) (2019)
- [Discrete Variational Autoencoders](https://mlsanity.com/paper/1609.02200/) (2016)
- [Reconstruction and Membership Inference Attacks against Generative Models](https://mlsanity.com/paper/1906.03006/) (2019)
- [Membership Inference Attacks by Exploiting Loss Trajectory](https://mlsanity.com/paper/2208.14933/) (2022)
- [ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models](https://mlsanity.com/paper/2102.02551/) (2021)
- [Node-Level Membership Inference Attacks Against Graph Neural Networks](https://mlsanity.com/paper/2102.05429/) (2021)
- [Membership Inference Attack on Graph Neural Networks](https://mlsanity.com/paper/2101.06570/) (2021)
- [Practical Blind Membership Inference Attack via Differential Comparisons](https://mlsanity.com/paper/2101.01341/) (2021)
- [How to Prove Your Model Belongs to You: A Blind-Watermark based Framework to Protect Intellectual Property of DNN](https://mlsanity.com/paper/1903.01743/) (2019)

## Built on by (most cited first, 30 of 69)

- [Extracting Training Data from Diffusion Models](https://mlsanity.com/paper/2301.13188/) (2023)
- [Text-to-image Diffusion Models in Generative AI: A Survey](https://mlsanity.com/paper/2303.07909/) (2023)
- [A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT](https://mlsanity.com/paper/2303.04226/) (2023)
- [SneakyPrompt: Jailbreaking Text-to-image Generative Models](https://mlsanity.com/paper/2305.12082/) (2023)
- [Are Diffusion Models Vulnerable to Membership Inference Attacks?](https://mlsanity.com/paper/2302.01316/) (2023)
- [Membership Inference Attacks against Diffusion Models](https://mlsanity.com/paper/2302.03262/) (2023)
- [Unlearning Concepts in Diffusion Model via Concept Domain Correction and Concept Preserving Gradient](https://mlsanity.com/paper/2405.15304/) (2024)
- [Blind Baselines Beat Membership Inference Attacks for Foundation Models](https://mlsanity.com/paper/2406.16201/) (2024)
- [On the Generalization Properties of Diffusion Models](https://mlsanity.com/paper/2311.01797/) (2023)
- [Safety at Scale: A Comprehensive Survey of Large Model and Agent Safety](https://mlsanity.com/paper/2502.05206/) (2025)
- [Prompt Stealing Attacks Against Text-to-Image Generation Models](https://mlsanity.com/paper/2302.09923/) (2023)
- [Membership Inference of Diffusion Models](https://mlsanity.com/paper/2301.09956/) (2023)
- [Membership Inference Attacks Against In-Context Learning](https://mlsanity.com/paper/2409.01380/) (2024)
- [White-box Membership Inference Attacks against Diffusion Models](https://mlsanity.com/paper/2308.06405/) (2023)
- [Towards More Realistic Membership Inference Attacks on Large Diffusion Models](https://mlsanity.com/paper/2306.12983/) (2023)
- [T2IAT: Measuring Valence and Stereotypical Biases in Text-to-Image Generation](https://mlsanity.com/paper/2306.00905/) (2023)
- [Practical Membership Inference Attacks Against Large-Scale Multi-Modal Models: A Pilot Study](https://mlsanity.com/paper/2310.00108/) (2023)
- [Membership Inference on Text-to-Image Diffusion Models via Conditional Likelihood Discrepancy](https://mlsanity.com/paper/2405.14800/) (2024)
- [Black-box Membership Inference Attacks against Fine-tuned Diffusion Models](https://mlsanity.com/paper/2312.08207/) (2023)
- [Attacks and Defenses for Generative Diffusion Models: A Comprehensive Survey](https://mlsanity.com/paper/2408.03400/) (2024)
- [Fair Text-to-Image Diffusion via Fair Mapping](https://mlsanity.com/paper/2311.17695/) (2023)
- [DP-LDMs: Differentially Private Latent Diffusion Models](https://mlsanity.com/paper/2305.15759/) (2023)
- [Bigger Isn't Always Memorizing: Early Stopping Overparameterized Diffusion Models](https://mlsanity.com/paper/2505.16959/) (2025)
- [SeqMIA: Sequential-Metric Based Membership Inference Attack](https://mlsanity.com/paper/2407.15098/) (2024)
- [Differentially Private Fine-Tuning of Diffusion Models](https://mlsanity.com/paper/2406.01355/) (2024)
- [Shake to Leak: Fine-tuning Diffusion Models Can Amplify the Generative Privacy Risk](https://mlsanity.com/paper/2403.09450/) (2024)
- [On the Trustworthiness Landscape of State-of-the-art Generative Models: A Survey and Outlook](https://mlsanity.com/paper/2307.16680/) (2023)
- [Towards Black-Box Membership Inference Attack for Diffusion Models](https://mlsanity.com/paper/2405.20771/) (2024)
- [A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models](https://mlsanity.com/paper/2307.12980/) (2023)
- [On the Limitations and Prospects of Machine Unlearning for Generative AI](https://mlsanity.com/paper/2408.00376/) (2024)

## Abstract (arXiv)

Text-to-image generation models have recently attracted unprecedented attention as they unlatch imaginative applications in all areas of life. However, developing such models requires huge amounts of data that might contain privacy-sensitive information, e.g., face identity. While privacy risks have been extensively demonstrated in the image classification and GAN generation domains, privacy risks in the text-to-image generation domain are largely unexplored. In this paper, we perform the first privacy analysis of text-to-image generation models through the lens of membership inference. Specifically, we propose three key intuitions about membership information and design four attack methodologies accordingly. We conduct comprehensive evaluations on two mainstream text-to-image generation models including sequence-to-sequence modeling and diffusion-based modeling. The empirical results show that all of the proposed attacks can achieve significant performance, in some cases even close to an accuracy of 1, and thus the corresponding risk is much more severe than that shown by existing membership inference attacks. We further conduct an extensive ablation study to analyze the factors that may affect the attack performance, which can guide developers and researchers to be alert to vulnerabilities in text-to-image generation models. All these findings indicate that our proposed attacks pose a realistic privacy threat to the text-to-image generation models.