# Nightshade: Prompt-Specific Poisoning Attacks on Text-to-Image Generative Models

arXiv: https://arxiv.org/abs/2310.13828 · HTML: https://mlsanity.com/paper/2310.13828/

## Key facts

- Cited by 75 papers in the map (data through 14 Oct 2026).
- Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security.
- Builds on 62 of its 115 references that are in the map.
- First posted on arXiv 20 Oct 2023.

## Placed in

- LLM safety and security / [Backdoor and data poisoning attacks](https://mlsanity.com/safety/backdoor-and-data-poisoning-attacks/)

## Builds on (most cited first, 30 of 62)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts](https://mlsanity.com/paper/2102.08981/) (2021)
- [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017)
- [Machine Unlearning](https://mlsanity.com/paper/1912.03817/) (2019)
- [Show and Tell: A Neural Image Caption Generator](https://mlsanity.com/paper/1411.4555/) (2014)
- [CogView: Mastering Text-to-Image Generation via Transformers](https://mlsanity.com/paper/2105.13290/) (2021)
- [AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks](https://mlsanity.com/paper/1711.10485/) (2017)
- [Score-based Generative Modeling in Latent Space](https://mlsanity.com/paper/2106.05931/) (2021)
- [Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networks](https://mlsanity.com/paper/1804.00792/) (2018)
- [Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering](https://mlsanity.com/paper/1811.03728/) (2018)
- [Understanding the Behaviour of Contrastive Loss](https://mlsanity.com/paper/2012.09740/) (2020)
- [Hidden Trigger Backdoor Attacks](https://mlsanity.com/paper/1910.00033/) (2019)
- [The Curse of Recursion: Training on Generated Data Makes Models Forget](https://mlsanity.com/paper/2305.17493/) (2023)
- [Descent-to-Delete: Gradient-Based Methods for Machine Unlearning](https://mlsanity.com/paper/2007.02923/) (2020)
- [DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesis](https://mlsanity.com/paper/1904.01310/) (2019)
- [Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2303.17591/) (2023)

## Built on by (most cited first, 30 of 75)

- [A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly](https://mlsanity.com/paper/2312.02003/) (2023)
- [GASLITEing the Retrieval: Exploring Vulnerabilities in Dense Embedding-based Search](https://mlsanity.com/paper/2412.20953/) (2024)
- [Data Poisoning in Deep Learning: A Survey](https://mlsanity.com/paper/2503.22759/) (2025)
- [Attack and defense techniques in large language models: A survey and new perspectives](https://mlsanity.com/paper/2505.00976/) (2025)
- [More Robots are Coming: Large Multimodal Models (ChatGPT) can Solve Visually Diverse Images of Parsons Problems](https://mlsanity.com/paper/2311.04926/) (2023)
- [Chain-of-Jailbreak Attack for Image Generation Models via Editing Step by Step](https://mlsanity.com/paper/2410.03869/) (2024)
- [Silent Branding Attack: Trigger-free Data Poisoning Attack on Text-to-Image Diffusion Models](https://mlsanity.com/paper/2503.09669/) (2025)
- [Disguised Copyright Infringement of Latent Diffusion Models](https://mlsanity.com/paper/2404.06737/) (2024)
- [BadVideo: Stealthy Backdoor Attack against Text-to-Video Generation](https://mlsanity.com/paper/2504.16907/) (2025)
- [Erased but Not Forgotten: How Backdoors Compromise Concept Erasure](https://mlsanity.com/paper/2504.21072/) (2025)
- [A LoRA is Worth a Thousand Pictures](https://mlsanity.com/paper/2412.12048/) (2024)
- [Algorithmic Collective Action with Two Collectives](https://mlsanity.com/paper/2505.00195/) (2025)
- [Understanding Implosion in Text-to-Image Generative Models](https://mlsanity.com/paper/2409.12314/) (2024)
- [Un-Straightening Generative AI: How Queer Artists Surface and Challenge the Normativity of Generative AI Models](https://mlsanity.com/paper/2503.09805/) (2025)
- [Enhancing Variational Autoencoders with Smooth Robust Latent Encoding](https://mlsanity.com/paper/2504.17219/) (2025)
- [Interpreting Large Text-to-Image Diffusion Models with Dictionary Learning](https://mlsanity.com/paper/2505.24360/) (2025)
- [MagicNaming: Consistent Identity Generation by Finding a "Name Space" in T2I Diffusion Models](https://mlsanity.com/paper/2412.14902/) (2024)
- [Improving Diffusion Models's Data-Corruption Resistance using Scheduled Pseudo-Huber Loss](https://mlsanity.com/paper/2403.16728/) (2024)
- [MaRVL-QA: A Benchmark for Mathematical Reasoning over Visual Landscapes](https://mlsanity.com/paper/2508.17180/) (2025)
- [Pixel Is Not a Barrier: An Effective Evasion Attack for Pixel-Domain Diffusion Models](https://mlsanity.com/paper/2408.11810/) (2024)
- [Buster: Implanting Semantic Backdoor into Text Encoder to Mitigate NSFW Content Generation](https://mlsanity.com/paper/2412.07249/) (2024)
- [BridgePure: Limited Protection Leakage Can Break Black-Box Data Protection](https://mlsanity.com/paper/2412.21061/) (2024)
- [Unveiling Unicode's Unseen Underpinnings in Undermining Authorship Attribution](https://mlsanity.com/paper/2508.15840/) (2025)
- [SafeGen-Bench: Benchmarking Safety in Image-Conditioned Text-to-Video Generation](https://mlsanity.com/paper/2606.01481/) (2026)
- [Combinational Backdoor Attack against Customized Text-to-Image Models](https://mlsanity.com/paper/2411.12389/) (2024)
- [Protective Perturbations against Unauthorized Data Usage in Diffusion-based Image Generation](https://mlsanity.com/paper/2412.18791/) (2024)
- [BadReward: Clean-Label Poisoning of Reward Models in Text-to-Image RLHF](https://mlsanity.com/paper/2506.03234/) (2025)
- [When and Where do Data Poisons Attack Textual Inversion?](https://mlsanity.com/paper/2507.10578/) (2025)
- [An h-space Based Adversarial Attack for Protection Against Few-shot Personalization](https://mlsanity.com/paper/2507.17554/) (2025)
- [HMARK: Radioactive Multi-Bit Semantic-Latent Watermarking for Diffusion Models](https://mlsanity.com/paper/2512.00094/) (2025)

## Abstract (arXiv)

Data poisoning attacks manipulate training data to introduce unexpected behaviors into machine learning models at training time. For text-to-image generative models with massive training datasets, current understanding of poisoning attacks suggests that a successful attack would require injecting millions of poison samples into their training pipeline. In this paper, we show that poisoning attacks can be successful on generative models. We observe that training data per concept can be quite limited in these models, making them vulnerable to prompt-specific poisoning attacks, which target a model's ability to respond to individual prompts. We introduce Nightshade, an optimized prompt-specific poisoning attack where poison samples look visually identical to benign images with matching text prompts. Nightshade poison samples are also optimized for potency and can corrupt an Stable Diffusion SDXL prompt in <100 poison samples. Nightshade poison effects "bleed through" to related concepts, and multiple attacks can composed together in a single prompt. Surprisingly, we show that a moderate number of Nightshade attacks can destabilize general features in a text-to-image generative model, effectively disabling its ability to generate meaningful images. Finally, we propose the use of Nightshade and similar tools as a last defense for content creators against web scrapers that ignore opt-out/do-not-crawl directives, and discuss possible implications for model trainers and content creators.