# UPAM: Unified Prompt Attack in Text-to-Image Generation Models Against Both Textual Filters and Visual Checkers

arXiv: https://arxiv.org/abs/2405.11336 · HTML: https://mlsanity.com/paper/2405.11336/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 46 references that are in the map.
- First posted on arXiv 18 May 2024.

## Builds on (most cited first, 27 of 27)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Scaling Autoregressive Models for Content-Rich Text-to-Image Generation](https://mlsanity.com/paper/2206.10789/) (2022)
- [ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision](https://mlsanity.com/paper/2102.03334/) (2021)
- [Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment](https://mlsanity.com/paper/1907.11932/) (2019)
- [AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks](https://mlsanity.com/paper/1711.10485/) (2017)
- [Adversarial Training for Free!](https://mlsanity.com/paper/1904.12843/) (2019)
- [Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models](https://mlsanity.com/paper/2211.05105/) (2022)
- [Label-Only Membership Inference Attacks](https://mlsanity.com/paper/2007.14321/) (2020)
- [Multimodal datasets: misogyny, pornography, and malignant stereotypes](https://mlsanity.com/paper/2110.01963/) (2021)
- [Stochastic Gradient Descent as Approximate Bayesian Inference](https://mlsanity.com/paper/1704.04289/) (2017)
- [Red-Teaming the Stable Diffusion Safety Filter](https://mlsanity.com/paper/2210.04610/) (2022)
- [StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks](https://mlsanity.com/paper/1710.10916/) (2017)
- [A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4](https://mlsanity.com/paper/2310.12321/) (2023)
- [BlackVIP: Black-Box Visual Prompting for Robust Transfer Learning](https://mlsanity.com/paper/2303.14773/) (2023)
- [Backdoor Defense with Machine Unlearning](https://mlsanity.com/paper/2201.09538/) (2022)
- [Label-Only Model Inversion Attacks via Boundary Repulsion](https://mlsanity.com/paper/2203.01925/) (2022)
- [Hard to Forget: Poisoning Attacks on Certified Machine Unlearning](https://mlsanity.com/paper/2109.08266/) (2021)
- [Discovering the Hidden Vocabulary of DALLE-2](https://mlsanity.com/paper/2206.00169/) (2022)
- [Adversarial Attacks on Image Generation With Made-Up Words](https://mlsanity.com/paper/2208.04135/) (2022)
- [The LogBarrier adversarial attack: making effective use of decision boundary information](https://mlsanity.com/paper/1903.10396/) (2019)
- [Continual Adversarial Defense](https://mlsanity.com/paper/2312.09481/) (2023)

## Built on by (most cited first, 12 of 12)

- [Perception-guided Jailbreak against Text-to-Image Models](https://mlsanity.com/paper/2408.10848/) (2024)
- [Jailbreak Attacks and Defenses against Multimodal Generative Models: A Survey](https://mlsanity.com/paper/2411.09259/) (2024)
- [A Comprehensive Survey on Concept Erasure in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2502.14896/) (2025)
- [Unified Prompt Attack Against Text-to-Image Generation Models](https://mlsanity.com/paper/2502.16423/) (2025)
- [DiffGraph: An Automated Agent-driven Model Merging Framework for In-the-Wild Text-to-Image Generation](https://mlsanity.com/paper/2603.20470/) (2026)
- [Antelope: Potent and Concealed Jailbreak Attack Strategy](https://mlsanity.com/paper/2412.08156/) (2024)
- [SafeGen-Bench: Benchmarking Safety in Image-Conditioned Text-to-Video Generation](https://mlsanity.com/paper/2606.01481/) (2026)
- [External Data Extraction Attacks against Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2510.02964/) (2025)
- [Security Risk of Misalignment between Text and Image in Multi-modal Model](https://mlsanity.com/paper/2510.26105/) (2025)
- [Dynamic Defense Profiling Enables Cognitive Jailbreak of Text-to-Image Models](https://mlsanity.com/paper/2607.17779/) (2026)
- [The Safeguard Worked. Is the LLM System Safer?](https://mlsanity.com/paper/2609.00519/) (2026)
- [Keep It CALM: Analyzing the Limits of Global Unsafety in Text-to-Image Generation](https://mlsanity.com/paper/2610.02300/) (2026)

## Abstract (arXiv)

Text-to-Image (T2I) models have raised security concerns due to their potential to generate inappropriate or harmful images. In this paper, we propose UPAM, a novel framework that investigates the robustness of T2I models from the attack perspective. Unlike most existing attack methods that focus on deceiving textual defenses, UPAM aims to deceive both textual and visual defenses in T2I models. UPAM enables gradient-based optimization, offering greater effectiveness and efficiency than previous methods. Given that T2I models might not return results due to defense mechanisms, we introduce a Sphere-Probing Learning (SPL) scheme to support gradient optimization even when no results are returned. Additionally, we devise a Semantic-Enhancing Learning (SEL) scheme to finetune UPAM for generating target-aligned images. Our framework also ensures attack stealthiness. Extensive experiments demonstrate UPAM's effectiveness and efficiency.