# Harnessing LLM to Attack LLM-Guarded Text-to-Image Models

arXiv: https://arxiv.org/abs/2312.07130 · HTML: https://mlsanity.com/paper/2312.07130/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Placed in Vision-language and text-to-image safety, one of 12 directions in LLM safety and security.
- Builds on 8 of its 46 references that are in the map.
- First posted on arXiv 12 Dec 2023.

## Placed in

- LLM safety and security / [Vision-language and text-to-image safety](https://mlsanity.com/safety/vision-language-and-text-to-image-safety/)

## Builds on (most cited first, 8 of 8)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [TextBugger: Generating Adversarial Text Against Real-world Applications](https://mlsanity.com/paper/1812.05271/) (2018)
- [SneakyPrompt: Jailbreaking Text-to-image Generative Models](https://mlsanity.com/paper/2305.12082/) (2023)
- [Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Models](https://mlsanity.com/paper/2305.13873/) (2023)
- [Black Box Adversarial Prompting for Foundation Models](https://mlsanity.com/paper/2302.04237/) (2023)
- [Adversarial Attacks on Image Generation With Made-Up Words](https://mlsanity.com/paper/2208.04135/) (2022)
- [An Automated Survey of Generative Artificial Intelligence: Large Language Models, Architectures, Protocols, and Applications](https://mlsanity.com/paper/2306.02781/) (2023)

## Built on by (most cited first, 12 of 12)

- [Safety at Scale: A Comprehensive Survey of Large Model and Agent Safety](https://mlsanity.com/paper/2502.05206/) (2025)
- [Reason2Attack: Jailbreaking Text-to-Image Models via LLM Reasoning](https://mlsanity.com/paper/2503.17987/) (2025)
- [Mind the Gesture: Evaluating AI Sensitivity to Culturally Offensive Non-Verbal Gestures](https://mlsanity.com/paper/2502.17710/) (2025)
- [PromptSafe: Gated Prompt Tuning for Safe Text-to-Image Generation](https://mlsanity.com/paper/2508.01272/) (2025)
- [PC2: Politically Controversial Content Generation via Jailbreaking Attacks on GPT-based Text-to-Image Models](https://mlsanity.com/paper/2601.05150/) (2026)
- [The Salami Slicing Threat: Exploiting Cumulative Risks in LLM Systems](https://mlsanity.com/paper/2604.11309/) (2026)
- [Finding a Wolf in Sheep's Clothing: Combating Adversarial Text-To-Image Prompts with Text Summarization](https://mlsanity.com/paper/2412.12212/) (2024)
- [PixJail: Self-Evolving Paper-to-Pipeline Reproduction for Text-to-Image Jailbreak Evaluation](https://mlsanity.com/paper/2606.24081/) (2026)
- [Mind the Gap: Zero-Query Jailbreaks via Filter-Generator Discrepancy in Text-to-Image Systems](https://mlsanity.com/paper/2608.00973/) (2026)
- [Multi2AV-Safety: Benchmarking Safety in Multimodal-to-Audio-Video Generation](https://mlsanity.com/paper/2608.26535/) (2026)
- [Jailbreaking Text-to-Image Models Through Cracks: Navigating Heterogeneous Safety Filters via Multi-Agent Debate](https://mlsanity.com/paper/2609.01168/) (2026)
- [RISE: Red-teaming via Iterative Strategy Evolution for Modern Text-to-Image Models](https://mlsanity.com/paper/2609.34920/) (2026)

## Abstract (arXiv)

To prevent Text-to-Image (T2I) models from generating unethical images, people deploy safety filters to block inappropriate drawing prompts. Previous works have employed token replacement to search adversarial prompts that attempt to bypass these filters, but they have become ineffective as nonsensical tokens fail semantic logic checks. In this paper, we approach adversarial prompts from a different perspective. We demonstrate that rephrasing a drawing intent into multiple benign descriptions of individual visual components can obtain an effective adversarial prompt. We propose a LLM-piloted multi-agent method named DACA to automatically complete intended rephrasing. Our method successfully bypasses the safety filters of DALL-E 3 and Midjourney to generate the intended images, achieving success rates of up to 76.7% and 64% in the one-time attack, and 98% and 84% in the re-use attack, respectively. We open-source our code and dataset on [this link](https://github.com/researchcode003/DACA).