mlsanityData to 14 Oct 2026

Unified Prompt Attack Against Text-to-Image Generation Models

arXiv 2502.16423 ↗First posted 23 Feb 2025cs.CVcited by 5

On the map: LLM safety and security → Vision-language and text-to-image safety

Abstract (arXiv)

Text-to-Image (T2I) models have advanced significantly, but their growing popularity raises security concerns due to their potential to generate harmful images. To address these issues, we propose UPAM, a novel framework to evaluate the robustness of T2I models from an attack perspective. Unlike prior methods that focus solely on textual defenses, UPAM unifies the attack on both textual and visual defenses. Additionally, it enables gradient-based optimization, overcoming reliance on enumeration for improved efficiency and effectiveness. To handle cases where T2I models block image outputs due to defenses, we introduce Sphere-Probing Learning (SPL) to enable optimization even without image results. Following SPL, our model bypasses defenses, inducing the generation of harmful content. To ensure semantic alignment with attacker intent, we propose Semantic-Enhancing Learning (SEL) for precise semantic control. UPAM also prioritizes the naturalness of adversarial prompts using In-context Naturalness Enhancement (INE), making them harder for human examiners to detect. Additionally, we address the issue of iterative queries--common in prior methods and easily detectable by API defenders--by introducing Transferable Attack Learning (TAL), allowing effective attacks with minimal queries. Extensive experiments validate UPAM's superiority in effectiveness, efficiency, naturalness, and low query detection rates.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Vision-language and text-to-image safety, one of 12 directions in LLM safety and security. Builds on 50 of its 84 references that are in the map. First posted on arXiv 23 Feb 2025.

Builds on

50 of 84 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models26 Jul
  2. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Models22 Jun
  3. 2023SneakyPrompt: Jailbreaking Text-to-image Generative Models20 May
  4. 2022Red-Teaming the Stable Diffusion Safety Filter3 Oct

Other references · 46

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  8. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  10. 2013Intriguing properties of neural networkscited by 5,155
  11. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  12. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
34 more
  1. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  2. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  3. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,318
  4. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
  5. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,230
  6. 2016Practical Black-Box Attacks against Machine Learningcited by 903
  7. 2020Fast is better than free: Revisiting adversarial trainingcited by 727
  8. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 685
  9. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 671
  10. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  11. 2019Adversarial Training for Free!cited by 643
  12. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 577
  13. 2016Delving into Transferable Adversarial Examples and Black-box Attackscited by 536
  14. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 529
  15. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 513
  16. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 498
  17. 2018Universal Sentence Encodercited by 489
  18. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 333
  19. 2021Multimodal datasets: misogyny, pornography, and malignant stereotypescited by 327
  20. 2017StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 229
  21. 2023MMA-Diffusion: MultiModal Attack on Diffusion Modelscited by 194
  22. 2024Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agentscited by 137
  23. 2023A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4cited by 81
  24. 2023BlackVIP: Black-Box Visual Prompting for Robust Transfer Learningcited by 70
  25. 2023How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defensescited by 66
  26. 2024T2VSafetyBench: Evaluating the Safety of Text-to-Video Generative Modelscited by 66
  27. 2022Backdoor Defense with Machine Unlearningcited by 65
  28. 2017Feature-Guided Black-Box Safety Testing of Deep Neural Networkscited by 49
  29. 2022Discovering the Hidden Vocabulary of DALLE-2cited by 43
  30. 2020Blacklight: Scalable Defense for Neural Networks against Query-Based Black-Box Attackscited by 37
  31. 2022Adversarial Attacks on Image Generation With Made-Up Wordscited by 32
  32. 2021Attack as the Best Defense: Nullifying Image-to-image Translation GANs via Limit-aware Adversarial Attackcited by 15
  33. 2024UPAM: Unified Prompt Attack in Text-to-Image Generation Models Against Both Textual Filters and Visual Checkerscited by 12
  34. 2019The LogBarrier adversarial attack: making effective use of decision boundary informationcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025PLA: Prompt Learning Attack against Text-to-Image Generative Models
  2. 2025Reason2Attack: Jailbreaking Text-to-Image Models via LLM Reasoning
  3. 2026DiSCO: Defending text-to-image generation through distribution-guided contrastive prompt optimization
  4. 2026DTVI: Dual-Stage Textual and Visual Intervention for Safe Text-to-Image Generation
  5. 2024GuardT2I: Defending Text-to-Image Models from Adversarial Prompts
  6. 2025Fooling the Watchers: Breaking AIGC Detectors via Semantic Prompt Attacks
  7. 2023Evaluating the Robustness of Text-to-image Diffusion Models against Real-world Attacks
  8. 2025AutoPrompt: Automated Red-Teaming of Text-to-Image Models via LLM-Driven Adversarial Prompts