# Practical, Generalizable and Robust Backdoor Attacks on Text-to-Image Diffusion Models

arXiv: https://arxiv.org/abs/2508.01605 · HTML: https://mlsanity.com/paper/2508.01605/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models.
- Builds on 38 of its 58 references that are in the map.
- First posted on arXiv 3 Aug 2025.

## Placed in

- Diffusion models / [Robustness, privacy and adversarial aspects](https://mlsanity.com/diffusion/robustness-privacy-and-adversarial-aspects/)
- LLM safety and security / [Backdoor and data poisoning attacks](https://mlsanity.com/safety/backdoor-and-data-poisoning-attacks/)

## Builds on (most cited first, 30 of 38)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017)
- [Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning](https://mlsanity.com/paper/1712.05526/) (2017)
- [Demystifying Prompts in Language Models via Perplexity Estimation](https://mlsanity.com/paper/2212.04037/) (2022)
- [How to Backdoor Diffusion Models?](https://mlsanity.com/paper/2212.05400/) (2022)
- [Text-to-Image Diffusion Models can be Easily Backdoored through Multimodal Data Poisoning](https://mlsanity.com/paper/2305.04175/) (2023)
- [TrojDiff: Trojan Attacks on Diffusion Models with Diverse Targets](https://mlsanity.com/paper/2303.05762/) (2023)
- [Nightshade: Prompt-Specific Poisoning Attacks on Text-to-Image Generative Models](https://mlsanity.com/paper/2310.13828/) (2023)
- [VillanDiffusion: A Unified Backdoor Attack Framework for Diffusion Models](https://mlsanity.com/paper/2306.06874/) (2023)
- [Rickrolling the Artist: Injecting Backdoors into Text Encoders for Text-to-Image Synthesis](https://mlsanity.com/paper/2211.02408/) (2022)
- [Democratizing Neural Machine Translation with OPUS-MT](https://mlsanity.com/paper/2212.01936/) (2022)
- [TextAttack: A Framework for Adversarial Attacks, Data Augmentation, and Adversarial Training in NLP](https://mlsanity.com/paper/2005.05909/) (2020)
- [Personalization as a Shortcut for Few-Shot Backdoor Attack against Text-to-Image Diffusion Models](https://mlsanity.com/paper/2305.10701/) (2023)
- [BAGM: A Backdoor Attack for Manipulating Text-to-Image Generative Models](https://mlsanity.com/paper/2307.16489/) (2023)
- [ReFACT: Updating Text-to-Image Models by Editing the Text Encoder](https://mlsanity.com/paper/2306.00738/) (2023)
- [Elijah: Eliminating Backdoors Injected in Diffusion Models via Distribution Shift](https://mlsanity.com/paper/2312.00050/) (2023)

## Built on by (most cited first, 1 of 1)

- [When One Modality Rules Them All: Backdoor Modality Collapse in Multimodal Diffusion Models](https://mlsanity.com/paper/2603.06508/) (2026)

## Abstract (arXiv)

Text-to-image diffusion models (T2I DMs) have achieved remarkable success in generating high-quality and diverse images from text prompts, yet recent studies have revealed their vulnerability to backdoor attacks. Existing attack methods suffer from critical limitations: 1) they rely on unnatural adversarial prompts that lack human readability and require massive poisoned data; 2) their effectiveness is typically restricted to specific models, lacking generalizability; and 3) they can be mitigated by recent backdoor defenses. To overcome these challenges, we propose a novel backdoor attack framework that achieves three key properties: 1) Practicality: Our attack requires only a few stealthy backdoor samples to generate arbitrary attacker-chosen target images, as well as ensuring high-quality image generation in benign scenarios. 2) Generalizability: The attack is applicable across multiple T2I DMs without requiring model-specific redesign. 3) Robustness: The attack remains effective against existing backdoor defenses and adaptive defenses. Our extensive experimental results on multiple T2I DMs demonstrate that with only 10 carefully crafted backdoored samples, our attack method achieves >90% attack success rate with negligible degradation in benign image generation quality. We also conduct human evaluation to validate our attack effectiveness. Furthermore, recent backdoor detection and mitigation methods, as well as adaptive defense tailored to our attack are not sufficiently effective, highlighting the pressing need for more robust defense mechanisms against the proposed attack.