mlsanityData to 7 Oct 2026

Nightshade: Prompt-Specific Poisoning Attacks on Text-to-Image Generative Models

arXiv 2310.13828 ↗First posted 20 Oct 2023cs.CR cs.AIcited by 72

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

Data poisoning attacks manipulate training data to introduce unexpected behaviors into machine learning models at training time. For text-to-image generative models with massive training datasets, current understanding of poisoning attacks suggests that a successful attack would require injecting millions of poison samples into their training pipeline. In this paper, we show that poisoning attacks can be successful on generative models. We observe that training data per concept can be quite limited in these models, making them vulnerable to prompt-specific poisoning attacks, which target a model's ability to respond to individual prompts. We introduce Nightshade, an optimized prompt-specific poisoning attack where poison samples look visually identical to benign images with matching text prompts. Nightshade poison samples are also optimized for potency and can corrupt an Stable Diffusion SDXL prompt in <100 poison samples. Nightshade poison effects "bleed through" to related concepts, and multiple attacks can composed together in a single prompt. Surprisingly, we show that a moderate number of Nightshade attacks can destabilize general features in a text-to-image generative model, effectively disabling its ability to generate meaningful images. Finally, we propose the use of Nightshade and similar tools as a last defense for content creators against web scrapers that ignore opt-out/do-not-crawl directives, and discuss possible implications for model trainers and content creators.

Key facts. Cited by 72 papers in the map (data through 7 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 62 of its 115 references that are in the map. First posted on arXiv 20 Oct 2023.

Builds on

62 of 115 references.

Closest ancestors · 7

Same direction, newest first

  1. 2023Text-to-Image Diffusion Models can be Easily Backdoored through Multimodal Data Poisoning7 May
  2. 2022How to Backdoor Diffusion Models?11 Dec
  3. 2022Pre-trained Encoders in Self-Supervised Learning Improve Secure and Privacy-preserving Supervised Learning6 Dec
  4. 2022CorruptEncoder: Data Poisoning based Backdoor Attacks to Contrastive Learning15 Nov
  5. 2022Indiscriminate Data Poisoning Attacks on Neural Networks19 Apr
  6. 2021Spinning Language Models: Risks of Propaganda-As-A-Service and Countermeasures9 Dec
  7. 2021Poison Forensics: Traceback of Data Poisoning Attacks in Neural Networks13 Oct

Other references · 55

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  7. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  8. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  9. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  10. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  11. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  12. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
38 more
  1. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  2. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  3. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  4. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
  5. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  6. 2019Machine Unlearningcited by 1,020
  7. 2014Show and Tell: A Neural Image Caption Generatorcited by 877
  8. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  9. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  10. 2021Score-based Generative Modeling in Latent Spacecited by 634
  11. 2018Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networkscited by 500
  12. 2018Detecting Backdoor Attacks on Deep Neural Networks by Activation Clusteringcited by 429
  13. 2020Understanding the Behaviour of Contrastive Losscited by 397
  14. 2019Hidden Trigger Backdoor Attackscited by 359
  15. 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 343
  16. 2019DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesiscited by 308
  17. 2020Descent-to-Delete: Gradient-Based Methods for Machine Unlearningcited by 303
  18. 2023Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Modelscited by 297
  19. 2023Poisoning Web-Scale Training Datasets is Practicalcited by 276
  20. 2023Poisoning Language Models During Instruction Tuningcited by 254
  21. 2023Glaze: Protecting Artists from Style Mimicry by Text-to-Image Modelscited by 200
  22. 2020BadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvementscited by 191
  23. 2015Large-scale Classification of Fine-Art Paintings: Learning The Right Metric on The Right Featurecited by 181
  24. 2020Blind Backdoors in Deep Learning Modelscited by 174
  25. 2020Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defensescited by 171
  26. 2023Adversarial Example Does Good: Preventing Painting Imitation from Diffusion Models via Adversarial Examplescited by 163
  27. 2023Raising the Cost of Malicious AI-Powered Image Editingcited by 160
  28. 2019Transferable Clean-Label Poisoning Attacks on Deep Neural Netscited by 129
  29. 2020Backdoor Attacks Against Deep Learning Systems in the Physical Worldcited by 129
  30. 2020You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completioncited by 112
  31. 2020Just How Toxic is Data Poisoning? A Unified Benchmark for Backdoor and Data Poisoning Attackscited by 109
  32. 2023Improving Multimodal Datasets with Image Captioningcited by 93
  33. 2021LowKey: Leveraging Adversarial Attacks to Protect Social Media Users from Facial Recognitioncited by 92
  34. 2020Intrinsic Certified Robustness of Bagging against Data Poisoning Attackscited by 90
  35. 2023TrojDiff: Trojan Attacks on Diffusion Models with Diverse Targetscited by 88
  36. 2019Defending Neural Backdoors via Generative Distribution Modelingcited by 82
  37. 2020On Certifying Robustness against Backdoor Attacks via Randomized Smoothingcited by 74
  38. 2020Bullseye Polytope: A Scalable Clean-Label Poisoning Attack with Improved Transferabilitycited by 65

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 13; 2025: 23; 2026: 33.

72 papers.

Sort:

By area:

  1. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 380
  2. 2024GASLITEing the Retrieval: Exploring Vulnerabilities in Dense Embedding-based Searchcited by 20
  3. 2025Data Poisoning in Deep Learning: A Surveycited by 20
  4. 2025Attack and defense techniques in large language models: A survey and new perspectivescited by 13
  5. 2024Chain-of-Jailbreak Attack for Image Generation Models via Editing Step by Stepcited by 11
  6. 2025Silent Branding Attack: Trigger-free Data Poisoning Attack on Text-to-Image Diffusion Modelscited by 11
  7. 2023More Robots are Coming: Large Multimodal Models (ChatGPT) can Solve Visually Diverse Images of Parsons Problemscited by 10
  8. 2024Disguised Copyright Infringement of Latent Diffusion Modelscited by 10
  9. 2025BadVideo: Stealthy Backdoor Attack against Text-to-Video Generationcited by 9
  10. 2025Erased but Not Forgotten: How Backdoors Compromise Concept Erasurecited by 9
  11. 2024A LoRA is Worth a Thousand Picturescited by 7
  12. 2025Algorithmic Collective Action with Two Collectivescited by 7
38 more
  1. 2024Understanding Implosion in Text-to-Image Generative Modelscited by 6
  2. 2025Un-Straightening Generative AI: How Queer Artists Surface and Challenge the Normativity of Generative AI Modelscited by 6
  3. 2025Enhancing Variational Autoencoders with Smooth Robust Latent Encodingcited by 6
  4. 2025Interpreting Large Text-to-Image Diffusion Models with Dictionary Learningcited by 6
  5. 2024MagicNaming: Consistent Identity Generation by Finding a "Name Space" in T2I Diffusion Modelscited by 5
  6. 2025MaRVL-QA: A Benchmark for Mathematical Reasoning over Visual Landscapescited by 4
  7. 2024Improving Diffusion Models's Data-Corruption Resistance using Scheduled Pseudo-Huber Losscited by 3
  8. 2024Pixel Is Not a Barrier: An Effective Evasion Attack for Pixel-Domain Diffusion Modelscited by 3
  9. 2024Buster: Implanting Semantic Backdoor into Text Encoder to Mitigate NSFW Content Generationcited by 3
  10. 2024BridgePure: Limited Protection Leakage Can Break Black-Box Data Protectioncited by 3
  11. 2025Unveiling Unicode's Unseen Underpinnings in Undermining Authorship Attributioncited by 3
  12. 2026SafeGen-Bench: Benchmarking Safety in Image-Conditioned Text-to-Video Generationcited by 3
  13. 2024Combinational Backdoor Attack against Customized Text-to-Image Modelscited by 2
  14. 2024Protective Perturbations against Unauthorized Data Usage in Diffusion-based Image Generationcited by 2
  15. 2025BadReward: Clean-Label Poisoning of Reward Models in Text-to-Image RLHFcited by 2
  16. 2025When and Where do Data Poisons Attack Textual Inversion?cited by 2
  17. 2025An h-space Based Adversarial Attack for Protection Against Few-shot Personalizationcited by 2
  18. 2025HMARK: Radioactive Multi-Bit Semantic-Latent Watermarking for Diffusion Modelscited by 2
  19. 2026To See is Not to Learn: Protecting Multimodal Data from Unauthorized Fine-Tuning of Large Vision-Language Modelcited by 2
  20. 2026Security of World-Model-Based Embodied AI: A Lifecycle of Threats, Defenses, and Evaluationcited by 2
  21. 2024Prevailing Research Areas for Music AI in the Era of Foundation Modelscited by 1
  22. 2025On the Feasibility of Poisoning Text-to-Image AI Models via Adversarial Mislabelingcited by 1
  23. 2025Backdoors in Conditional Diffusion: Threats to Responsible Synthetic Data Pipelinescited by 1
  24. 2025Practical, Generalizable and Robust Backdoor Attacks on Text-to-Image Diffusion Modelscited by 1
  25. 2026Towards Poisoning Robustness Certification for Natural Language Generationcited by 1
  26. 2026Off-The-Shelf Image-to-Image Models Are All You Need To Defeat Image Protection Schemescited by 1
  27. 2026Immunizing 3D Gaussian Generative Models Against Unauthorized Fine-Tuning via Attribute-Space Trapscited by 1
  28. 2026Safeguarding Text-to-Image Generative Models Against Unauthorized Knowledge Distillationcited by 1
  29. 2023Trustworthy Large Models in Vision: A Surveycited by 0
  30. 2025What's Pulling the Strings? Evaluating Integrity and Attribution in AI Training and Inference through Concept Shiftcited by 0
  31. 2025Adversarial Attacks in Multimodal Systems: A Practitioner's Surveycited by 0
  32. 2025Threat Modeling for AI: The Case for an Asset-Centric Approachcited by 0
  33. 2025Attacks on Approximate Caches in Text-to-Image Diffusion Modelscited by 0
  34. 2025Adversarially-Aware Architecture Design for Robust Medical AI Systemscited by 0
  35. 2026Backdoor Sentinel: Detecting and Detoxifying Backdoors in Diffusion Models via Temporal Noise Consistencycited by 0
  36. 2026DICE: Disentangling Artist Style from Content via Contrastive Subspace Decomposition in Diffusion Modelscited by 0
  37. 2026When Backdoors Go Beyond Triggers: Semantic Drift in Diffusion Models Under Encoder Attackscited by 0
  38. 2026How Professional Visual Artists are Negotiating Generative AI in the Workplacecited by 0

50 most cited of 72 All 72 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Associative Poisoning to Generative Machine Learning
  2. 2026TraceGuard: Adaptive Multimodal Poison Filtering through Cross-Feature Rank Agreement
  3. 2024Shadowcast: Stealthy Data Poisoning Attacks Against Vision-Language Models
  4. 2023Pick your Poison: Undetectability versus Robustness in Data Poisoning Attacks
  5. 2025System Prompt Poisoning: Persistent Attacks on Large Language Models Beyond User Injection
  6. 2023From Trojan Horses to Castle Walls: Unveiling Bilateral Data Poisoning Effects in Diffusion Models
  7. 2023Adversarial Nibbler: A Data-Centric Challenge for Improving the Safety of Text-to-Image Models
  8. 2023Stable Diffusion is Unstable