mlsanityData to 7 Oct 2026

The Stronger the Diffusion Model, the Easier the Backdoor: Data Poisoning to Induce Copyright Breaches Without Adjusting Finetuning Pipeline

arXiv 2401.04136 ↗First posted 7 Jan 2024cs.CR cs.AIcited by 33

On the map: Diffusion models → Robustness, privacy and adversarial aspects

also in 1 more
Abstract (arXiv)

The commercialization of text-to-image diffusion models (DMs) brings forth potential copyright concerns. Despite numerous attempts to protect DMs from copyright issues, the vulnerabilities of these solutions are underexplored. In this study, we formalized the Copyright Infringement Attack on generative AI models and proposed a backdoor attack method, SilentBadDiffusion, to induce copyright infringement without requiring access to or control over training processes. Our method strategically embeds connections between pieces of copyrighted information and text references in poisoning data while carefully dispersing that information, making the poisoning data inconspicuous when integrated into a clean dataset. Our experiments show the stealth and efficacy of the poisoning data. When given specific text prompts, DMs trained with a poisoning ratio of 0.20% can produce copyrighted images. Additionally, the results reveal that the more sophisticated the DMs are, the easier the success of the attack becomes. These findings underline potential pitfalls in the prevailing copyright protection strategies and underscore the necessity for increased scrutiny to prevent the misuse of DMs.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models. Builds on 49 of its 71 references that are in the map. First posted on arXiv 7 Jan 2024.

Builds on

49 of 71 references.

Closest ancestors · 9

Same direction, newest first

  1. 2023DiffusionShield: A Watermark for Copyright Protection against Generative Diffusion Models25 May
  2. 2023Text-to-Image Diffusion Models can be Easily Backdoored through Multimodal Data Poisoning7 May
  3. 2023Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Models30 Mar
  4. 2023Ablating Concepts in Text-to-Image Diffusion Models23 Mar
  5. 2023A Recipe for Watermarking Diffusion Models17 Mar
  6. 2023Erasing Concepts from Diffusion Models13 Mar
  7. 2023TrojDiff: Trojan Attacks on Diffusion Models with Diverse Targets10 Mar
  8. 2023Extracting Training Data from Diffusion Models30 Jan
  9. 2022How to Backdoor Diffusion Models?11 Dec

Other references · 40

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  7. 2023Segment Anythingcited by 8,158
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  9. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  10. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  11. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  12. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
28 more
  1. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  2. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  3. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  4. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  5. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  6. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  7. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,036
  8. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 505
  9. 2022Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Modelscited by 418
  10. 2022Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Modelscited by 331
  11. 2023Poisoning Web-Scale Training Datasets is Practicalcited by 276
  12. 2022Cold Diffusion: Inverting Arbitrary Image Transforms Without Noisecited by 275
  13. 2023Understanding and Mitigating Copying in Diffusion Modelscited by 205
  14. 2023Generalization in diffusion models arises from geometry-adaptive harmonic representationscited by 179
  15. 2022A Self-Supervised Descriptor for Image Copy Detectioncited by 175
  16. 2021Towards Faster and Stabilized GAN Training for High-fidelity Few-shot Image Synthesiscited by 140
  17. 2023On Memorization in Diffusion Modelscited by 122
  18. 2020When is Memorization of Irrelevant Training Data Necessary for High-Accuracy Learning?cited by 102
  19. 2022Soft Diffusion: Score Matching for General Corruptionscited by 101
  20. 2023On Provable Copyright Protection for Generative Modelscited by 97
  21. 2023How Does Information Bottleneck Help Deep Learning?cited by 92
  22. 2023Efficient Dataset Distillation via Minimax Diffusioncited by 81
  23. 2022Untargeted Backdoor Watermark: Towards Harmless and Stealthy Dataset Copyright Protectioncited by 80
  24. 2023Domain Watermark: Effective and Harmless Dataset Copyright Protection is Closed at Handcited by 52
  25. 2023Did You Train on My Dataset? Towards Public Dataset Protection with Clean-Label Backdoor Watermarkingcited by 37
  26. 2024Can AI Be as Creative as Humans?cited by 17
  27. 2023Safe-CLIP: Removing NSFW Concepts from Vision-and-Language Modelscited by 11
  28. 2021Controllable Gradient Item Retrievalcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 12; 2025: 13; 2026: 8.

33 papers.

Sort:

By area:

  1. 2024Attacks and Defenses for Generative Diffusion Models: A Comprehensive Surveycited by 32
  2. 2024UFID: A Unified Framework for Input-level Backdoor Detection on Diffusion Modelscited by 14
  3. 2024Evaluating and Mitigating IP Infringement in Visual Generative AIcited by 13
  4. 2024Concept Arithmetics for Circumventing Concept Inhibition in Diffusion Modelscited by 11
  5. 2024Replication in Visual Diffusion Models: A Survey and Outlookcited by 11
  6. 2025Silent Branding Attack: Trigger-free Data Poisoning Attack on Text-to-Image Diffusion Modelscited by 11
  7. 2024ProTIP: Probabilistic Robustness Verification on Text-to-Image Diffusion Models against Stochastic Perturbationcited by 10
  8. 2025Red-Teaming Text-to-Image Systems by Rule-based Preference Modelingcited by 8
  9. 2025A Systematic Review of Poisoning Attacks Against Large Language Modelscited by 8
  10. 2025BackdoorDM: A Comprehensive Benchmark for Backdoor Learning on Diffusion Modelcited by 7
  11. 2025Winter Soldier: Backdooring Language Models at Pre-Training with Indirect Data Poisoningcited by 6
  12. 2024CopyrightShield: Enhancing Diffusion Model Security against Copyright Infringement Attackscited by 5
21 more
  1. 2025MixBridge: Heterogeneous Image-to-Image Backdoor Attack through Mixture of Schr"odinger Bridgescited by 5
  2. 2024Perturb and Recover: Fine-tuning for Effective Backdoor Removal from CLIPcited by 4
  3. 2025CopyJudge: Automated Copyright Infringement Identification and Mitigation in Text-to-Image Diffusion Modelscited by 4
  4. 2024The last Dance : Robust backdoor attack via diffusion models and bayesian approachcited by 3
  5. 2024Improving Diffusion Models's Data-Corruption Resistance using Scheduled Pseudo-Huber Losscited by 3
  6. 2024Trading Devil: Robust backdoor attack via Stochastic investment models and Bayesian approachcited by 3
  7. 2025A Dual-Purpose Framework for Backdoor Defense and Backdoor Amplification in Diffusion Modelscited by 3
  8. 2025PoisonedParrot: Subtle Data Poisoning Attacks to Elicit Copyright-Infringing Content from Large Language Modelscited by 3
  9. 2025A Linear Approach to Data Poisoningcited by 2
  10. 2025ME: Trigger Element Combination Backdoor Attack on Copyright Infringementcited by 2
  11. 2024TrojanEdit: Multimodal Backdoor Attack Against Image Editing Modelcited by 1
  12. 2026Safety-Efficacy Trade Off: Robustness against Data-Poisoningcited by 1
  13. 2025TWIG: Two-Step Image Generation using Segmentation Masks in Diffusion Modelscited by 0
  14. 2025Attacks on Approximate Caches in Text-to-Image Diffusion Modelscited by 0
  15. 2026A Unified Perspective on Adversarial Membership Manipulation in Vision Modelscited by 0
  16. 2026Yes, But Not Always. Generative AI Needs Nuanced Opt-incited by 0
  17. 2026Beyond the False Trade-off: Adaptive EWC for Stealthy and Generalizable T2I Backdoorscited by 0
  18. 2026Awakening the Hydra: Stabilizing Multi-Concept Backdoor Injection in Text-to-Image Diffusion Modelscited by 0
  19. 2026TooBad: Backdoor Diffusion Models with Ultra-Low Poison Rate and Imperceptible Triggercited by 0
  20. 2026Robust Watermarks Meet Backdoored Models: Evading Diffusion Semantic Watermarks via Stealthy Backdoorcited by 0
  21. 2026BadAction: Backdoor Attacks on Interactive Video Generation via Action-Guided Triggerscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023On Copyright Risks of Text-to-Image Diffusion Models
  2. 2024A Dataset and Benchmark for Copyright Infringement Unlearning from Text-to-Image Diffusion Models
  3. 2024CopyrightMeter: Revisiting Copyright Protection in Text-to-image Models
  4. 2024RLCP: A Reinforcement Learning-based Copyright Protection Method for Text-to-Image Diffusion Model
  5. 2025Copyright Infringement Detection in Text-to-Image Diffusion Models via Differential Privacy
  6. 2025Practical, Generalizable and Robust Backdoor Attacks on Text-to-Image Diffusion Models
  7. 2026Attention to Neural Plagiarism: Diffusion Models Can Plagiarize Your Copyrighted Images!
  8. 2024Disguised Copyright Infringement of Latent Diffusion Models