# Black-box Membership Inference Attacks against Fine-tuned Diffusion Models

arXiv: https://arxiv.org/abs/2312.08207 · HTML: https://mlsanity.com/paper/2312.08207/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models.
- Builds on 52 of its 81 references that are in the map.
- First posted on arXiv 13 Dec 2023.

## Placed in

- Diffusion models / [Robustness, privacy and adversarial aspects](https://mlsanity.com/diffusion/robustness-privacy-and-adversarial-aspects/)
- LLM safety and security / [Membership inference attacks](https://mlsanity.com/safety/membership-inference-attacks/)

## Builds on (most cited first, 30 of 52)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [Vector Quantized Diffusion Model for Text-to-Image Synthesis](https://mlsanity.com/paper/2111.14822/) (2021)
- [Extracting Training Data from Diffusion Models](https://mlsanity.com/paper/2301.13188/) (2023)
- [ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models](https://mlsanity.com/paper/1806.01246/) (2018)
- [Label-Only Membership Inference Attacks](https://mlsanity.com/paper/2007.14321/) (2020)
- [Systematic Evaluation of Privacy Risks of Machine Learning Models](https://mlsanity.com/paper/2003.10595/) (2020)
- [WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learning](https://mlsanity.com/paper/2103.01913/) (2021)
- [White-box vs Black-box: Bayes Optimal Strategies for Membership Inference](https://mlsanity.com/paper/1908.11229/) (2019)
- [EfficientFormer: Vision Transformers at MobileNet Speed](https://mlsanity.com/paper/2206.01191/) (2022)
- [Glaze: Protecting Artists from Style Mimicry by Text-to-Image Models](https://mlsanity.com/paper/2302.04222/) (2023)

## Built on by (most cited first, 30 of 35)

- [Safety at Scale: A Comprehensive Survey of Large Model and Agent Safety](https://mlsanity.com/paper/2502.05206/) (2025)
- [Membership Inference on Text-to-Image Diffusion Models via Conditional Likelihood Discrepancy](https://mlsanity.com/paper/2405.14800/) (2024)
- [Attacks and Defenses for Generative Diffusion Models: A Comprehensive Survey](https://mlsanity.com/paper/2408.03400/) (2024)
- [Towards Reliable Verification of Unauthorized Data Usage in Personalized Text-to-Image Diffusion Models](https://mlsanity.com/paper/2410.10437/) (2024)
- [Who Stole Your Data? A Method for Detecting Unauthorized RAG Theft](https://mlsanity.com/paper/2510.07728/) (2025)
- [Towards Black-Box Membership Inference Attack for Diffusion Models](https://mlsanity.com/paper/2405.20771/) (2024)
- [Replication in Visual Diffusion Models: A Survey and Outlook](https://mlsanity.com/paper/2408.00001/) (2024)
- [Privacy-Preserving Low-Rank Adaptation against Membership Inference Attacks for Latent Diffusion Models](https://mlsanity.com/paper/2402.11989/) (2024)
- [ArtistAuditor: Auditing Artist Style Pirate in Text-to-Image Generation Models](https://mlsanity.com/paper/2504.13061/) (2025)
- [Membership Inference Attacks Against Fine-tuned Diffusion Language Models](https://mlsanity.com/paper/2601.20125/) (2026)
- [CPSample: Classifier Protected Sampling for Guarding Training Data During Diffusion](https://mlsanity.com/paper/2409.07025/) (2024)
- [Dual-Model Defense: Safeguarding Diffusion Models from Membership Inference Attacks through Disjoint Data Splitting](https://mlsanity.com/paper/2410.16657/) (2024)
- [CompLeak: Deep Learning Model Compression Exacerbates Privacy Leakage](https://mlsanity.com/paper/2507.16872/) (2025)
- [Responsible Diffusion: A Comprehensive Survey on Safety, Ethics, and Trust in Diffusion Models](https://mlsanity.com/paper/2509.22723/) (2025)
- [Membership Inference Attacks on Tokenizers of Large Language Models](https://mlsanity.com/paper/2510.05699/) (2025)
- [Quantifying the Privacy Implications of High-Fidelity Synthetic Network Traffic](https://mlsanity.com/paper/2511.20497/) (2025)
- [In-Context Probing for Membership Inference in Fine-Tuned Language Models](https://mlsanity.com/paper/2512.16292/) (2025)
- [Inference Attacks Against Graph Generative Diffusion Models](https://mlsanity.com/paper/2601.03701/) (2026)
- [VLALeaks: Membership Inference Attacks against Vision-Language-Action Models](https://mlsanity.com/paper/2606.15165/) (2026)
- [Identity-Focused Inference and Extraction Attacks on Diffusion Models](https://mlsanity.com/paper/2410.10177/) (2024)
- [Enhancing Membership Inference Attacks on Diffusion Models from a Frequency-Domain Perspective](https://mlsanity.com/paper/2505.20955/) (2025)
- [SoK: Can Synthetic Images Replace Real Data? A Survey of Utility and Privacy of Synthetic Image Generation](https://mlsanity.com/paper/2506.19360/) (2025)
- [Membership Inference Attack with Partial Features](https://mlsanity.com/paper/2508.06244/) (2025)
- [EnTruth: Enhancing the Traceability of Unauthorized Dataset Usage in Text-to-image Diffusion Models with Minimal and Robust Alterations](https://mlsanity.com/paper/2406.13933/) (2024)
- [Exploiting Watermark-Based Defense Mechanisms in Text-to-Image Diffusion Models for Unauthorized Data Usage](https://mlsanity.com/paper/2411.15367/) (2024)
- [Noise as a Probe: Membership Inference Attacks on Diffusion Models Leveraging Initial Noise](https://mlsanity.com/paper/2601.21628/) (2026)
- [The Silent Brush: Evaluating Artistic Style Leakage in AI Art Generation](https://mlsanity.com/paper/2605.17500/) (2026)
- [Black-box Membership Inference Attacks on the Pre-training Data of Image-generation Models](https://mlsanity.com/paper/2605.27020/) (2026)
- [ImageAuditor: Membership Inference Attack against Image-based Retrieval-Augmented Generation](https://mlsanity.com/paper/2606.03354/) (2026)
- [One Framework for All: Cross-Modal Membership Inference for Generative Models](https://mlsanity.com/paper/2607.04339/) (2026)

## Abstract (arXiv)

With the rapid advancement of diffusion-based image-generative models, the quality of generated images has become increasingly photorealistic. Moreover, with the release of high-quality pre-trained image-generative models, a growing number of users are downloading these pre-trained models to fine-tune them with downstream datasets for various image-generation tasks. However, employing such powerful pre-trained models in downstream tasks presents significant privacy leakage risks. In this paper, we propose the first reconstruction-based membership inference attack framework, tailored for recent diffusion models, and in the more stringent black-box access setting. Considering four distinct attack scenarios and three types of attacks, this framework is capable of targeting any popular conditional generator model, achieving high precision, evidenced by an impressive AUC of 0.95.