Unified Prompt Attack Against Text-to-Image Generation Models
On the map: LLM safety and security → Vision-language and text-to-image safety
Abstract (arXiv)
Text-to-Image (T2I) models have advanced significantly, but their growing popularity raises security concerns due to their potential to generate harmful images. To address these issues, we propose UPAM, a novel framework to evaluate the robustness of T2I models from an attack perspective. Unlike prior methods that focus solely on textual defenses, UPAM unifies the attack on both textual and visual defenses. Additionally, it enables gradient-based optimization, overcoming reliance on enumeration for improved efficiency and effectiveness. To handle cases where T2I models block image outputs due to defenses, we introduce Sphere-Probing Learning (SPL) to enable optimization even without image results. Following SPL, our model bypasses defenses, inducing the generation of harmful content. To ensure semantic alignment with attacker intent, we propose Semantic-Enhancing Learning (SEL) for precise semantic control. UPAM also prioritizes the naturalness of adversarial prompts using In-context Naturalness Enhancement (INE), making them harder for human examiners to detect. Additionally, we address the issue of iterative queries--common in prior methods and easily detectable by API defenders--by introducing Transferable Attack Learning (TAL), allowing effective attacks with minimal queries. Extensive experiments validate UPAM's superiority in effectiveness, efficiency, naturalness, and low query detection rates.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Vision-language and text-to-image safety, one of 12 directions in LLM safety and security. Builds on 50 of its 84 references that are in the map. First posted on arXiv 23 Feb 2025.
Builds on
50 of 84 references.
Closest ancestors · 4
Same direction, newest first
- 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models26 Jul
- 2023Visual Adversarial Examples Jailbreak Aligned Large Language Models22 Jun
- 2023SneakyPrompt: Jailbreaking Text-to-image Generative Models20 May
- 2022Red-Teaming the Stable Diffusion Safety Filter3 Oct
Other references · 46
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2014Explaining and Harnessing Adversarial Examplescited by 7,364
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
- 2013Intriguing properties of neural networkscited by 5,155
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
- 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
34 more
- 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
- 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
- 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,318
- 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
- 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,230
- 2016Practical Black-Box Attacks against Machine Learningcited by 903
- 2020Fast is better than free: Revisiting adversarial trainingcited by 727
- 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 685
- 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 671
- 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
- 2019Adversarial Training for Free!cited by 643
- 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 577
- 2016Delving into Transferable Adversarial Examples and Black-box Attackscited by 536
- 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 529
- 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 513
- 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 498
- 2018Universal Sentence Encodercited by 489
- 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 333
- 2021Multimodal datasets: misogyny, pornography, and malignant stereotypescited by 327
- 2017StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 229
- 2023MMA-Diffusion: MultiModal Attack on Diffusion Modelscited by 194
- 2024Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agentscited by 137
- 2023A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4cited by 81
- 2023BlackVIP: Black-Box Visual Prompting for Robust Transfer Learningcited by 70
- 2023How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defensescited by 66
- 2024T2VSafetyBench: Evaluating the Safety of Text-to-Video Generative Modelscited by 66
- 2022Backdoor Defense with Machine Unlearningcited by 65
- 2017Feature-Guided Black-Box Safety Testing of Deep Neural Networkscited by 49
- 2022Discovering the Hidden Vocabulary of DALLE-2cited by 43
- 2020Blacklight: Scalable Defense for Neural Networks against Query-Based Black-Box Attackscited by 37
- 2022Adversarial Attacks on Image Generation With Made-Up Wordscited by 32
- 2021Attack as the Best Defense: Nullifying Image-to-image Translation GANs via Limit-aware Adversarial Attackcited by 15
- 2024UPAM: Unified Prompt Attack in Text-to-Image Generation Models Against Both Textual Filters and Visual Checkerscited by 12
- 2019The LogBarrier adversarial attack: making effective use of decision boundary informationcited by 7
Built on by
Citing papers per year: 2025: 3; 2026: 2.
5 papers.
- 2025Erasing Concepts, Steering Generations: A Comprehensive Survey of Concept Suppressioncited by 10
- 2025NSFW-Classifier Guided Prompt Sanitization for Safe Text-to-Image Generationcited by 3
- 2025ZIUM: Zero-Shot Intent-Aware Adversarial Attack on Unlearned Modelscited by 2
- 2026MicroscopyMatching: Towards a Ready-to-use Framework for Microscopy Image Analysis in Diverse Conditionscited by 0
- 2026UniNDM: A Unified Noise-driven Detection and Mitigation Framework Against Sexual Content in Text-to-Image Generationcited by 0
- 2026UniNDM: A Unified Noise-driven Detection and Mitigation Framework Against Sexual Content in Text-to-Image Generation18 Jul 2026
- 2026MicroscopyMatching: Towards a Ready-to-use Framework for Microscopy Image Analysis in Diverse Conditions14 May 2026
- 2025ZIUM: Zero-Shot Intent-Aware Adversarial Attack on Unlearned Models29 Jul 2025
- 2025NSFW-Classifier Guided Prompt Sanitization for Safe Text-to-Image Generation23 Jun 2025
- 2025Erasing Concepts, Steering Generations: A Comprehensive Survey of Concept Suppression26 May 2025
Similar, not cited
Close in wording, no citation either way
- 2025PLA: Prompt Learning Attack against Text-to-Image Generative Models
- 2025Reason2Attack: Jailbreaking Text-to-Image Models via LLM Reasoning
- 2026DiSCO: Defending text-to-image generation through distribution-guided contrastive prompt optimization
- 2026DTVI: Dual-Stage Textual and Visual Intervention for Safe Text-to-Image Generation
- 2024GuardT2I: Defending Text-to-Image Models from Adversarial Prompts
- 2025Fooling the Watchers: Breaking AIGC Detectors via Semantic Prompt Attacks
- 2023Evaluating the Robustness of Text-to-image Diffusion Models against Real-world Attacks
- 2025AutoPrompt: Automated Red-Teaming of Text-to-Image Models via LLM-Driven Adversarial Prompts