mlsanityData to 7 Oct 2026

Shake to Leak: Fine-tuning Diffusion Models Can Amplify the Generative Privacy Risk

arXiv 2403.09450 ↗First posted 14 Mar 2024cs.LG cs.CRcited by 17

On the map: Diffusion models → Robustness, privacy and adversarial aspects

Abstract (arXiv)

While diffusion models have recently demonstrated remarkable progress in generating realistic images, privacy risks also arise: published models or APIs could generate training images and thus leak privacy-sensitive training information. In this paper, we reveal a new risk, Shake-to-Leak (S2L), that fine-tuning the pre-trained models with manipulated data can amplify the existing privacy risks. We demonstrate that S2L could occur in various standard fine-tuning strategies for diffusion models, including concept-injection methods (DreamBooth and Textual Inversion) and parameter-efficient methods (LoRA and Hypernetwork), as well as their combinations. In the worst case, S2L can amplify the state-of-the-art membership inference attack (MIA) on diffusion models by 5.4% (absolute difference) AUC and can increase extracted private samples from almost 0 samples to 15.8 samples on average per target domain. This discovery underscores that the privacy risk with diffusion models is even more severe than previously recognized. Codes are available at https://github.com/VITA-Group/Shake-to-Leak.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models. Builds on 28 of its 37 references that are in the map. First posted on arXiv 14 Mar 2024.

Builds on

28 of 37 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023Differentially Private Diffusion Models Generate Useful Synthetic Images27 Feb
  2. 2023Are Diffusion Models Vulnerable to Membership Inference Attacks?2 Feb
  3. 2023Extracting Training Data from Diffusion Models30 Jan
  4. 2023Membership Inference of Diffusion Models24 Jan

Other references · 24

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  8. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  9. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  10. 2016Deep Learning with Differential Privacycited by 2,682
  11. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  12. 2021Membership Inference Attacks From First Principlescited by 839
12 more
  1. 2022Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Modelscited by 418
  2. 2021Differentially Private Fine-tuning of Language Modelscited by 392
  3. 2022Inversion-Based Style Transfer with Diffusion Modelscited by 290
  4. 2022Diffusion Models for Medical Image Analysis: A Comprehensive Surveycited by 269
  5. 2022Brain Imaging Generation with Latent Diffusion Modelscited by 267
  6. 2023Flocks of Stochastic Parrots: Differentially Private Prompt Learning for Large Language Modelscited by 80
  7. 2022Membership Inference Attacks Against Text-to-image Generation Modelscited by 66
  8. 2022Text-Guided Synthesis of Artistic Images with Retrieval-Augmented Diffusion Modelscited by 63
  9. 2023DP-OPT: Make Large Language Model Your Privacy-Preserving Prompt Engineercited by 56
  10. 2022Bayesian Estimation of Differential Privacycited by 31
  11. 2017Semi-Adversarial Networks: Convolutional Autoencoders for Imparting Privacy to Face Imagescited by 20
  12. 2023TMI! Finetuned Models Leak Private Information from their Pretraining Datacited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 12; 2025: 1; 2026: 4.

17 papers.

Sort:

By area:

  1. 2024Synthetic Face Datasets Generation via Latent Space Exploration from Brownian Identity Diffusioncited by 12
  2. 2024Replication in Visual Diffusion Models: A Survey and Outlookcited by 11
  3. 2024Trustworthy Text-to-Image Diffusion Models: A Timely and Focused Surveycited by 6
  4. 2024SoK: Blockchain-Based Decentralized AI (DeAI)cited by 6
  5. 2024Exactly Minimax-Optimal Locally Differentially Private Samplingcited by 5
  6. 2024A Survey on Privacy Attacks Against Digital Twin Systems in AI-Roboticscited by 3
  7. 2024Dual-Model Defense: Safeguarding Diffusion Models from Membership Inference Attacks through Disjoint Data Splittingcited by 3
  8. 2024Unveiling Synthetic Faces: How Synthetic Datasets Can Expose Real Identitiescited by 3
  9. 2025AugGen: Synthetic Augmentation using Diffusion Models Can Improve Recognitioncited by 3
  10. 2024Hide and Seek: How Does Watermarking Impact Face Recognition?cited by 2
  11. 2024Differentially Private Kernel Density Estimationcited by 2
  12. 2024Bridging Today and the Future of Humanity: AI Safety in 2024 and Beyondcited by 1
5 more
  1. 2024LoyalDiffusion: A Diffusion Model Guarding Against Data Replicationcited by 1
  2. 2026Evaluation without Generation: Non-Generative Assessment of Harmful Model Specialization with Applications to CSAMcited by 1
  3. 2026Noise as a Probe: Membership Inference Attacks on Diffusion Models Leveraging Initial Noisecited by 0
  4. 2026POPS: Recovering Unlearned Multi-Modality Knowledge in MLLMs with Prompt-Optimized Parameter Shakingcited by 0
  5. 2026PROVE: Training-Free Prompt Recovery using Verifiable Evidencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Differentially Private Fine-Tuning of Diffusion Models
  2. 2026Adaptive Diffusion Freezing: Privacy-preserving Diffusion Models Against Membership Inference Attacks
  3. 2023Data Forensics in Diffusion Models: A Systematic Analysis of Membership Privacy
  4. 2023Black-box Membership Inference Attacks against Fine-tuned Diffusion Models
  5. 2023Towards More Realistic Membership Inference Attacks on Large Diffusion Models
  6. 2023Privacy Threats in Stable Diffusion Models
  7. 2025On the MIA Vulnerability Gap Between Private GANs and Diffusion Models
  8. 2024Model Will Tell: Training Membership Inference for Diffusion Models