mlsanityData to 14 Oct 2026

Perception-guided Jailbreak against Text-to-Image Models

arXiv 2408.10848 ↗First posted 20 Aug 2024cs.CVcited by 40

On the map: LLM safety and security → Vision-language and text-to-image safety

Abstract (arXiv)

In recent years, Text-to-Image (T2I) models have garnered significant attention due to their remarkable advancements. However, security concerns have emerged due to their potential to generate inappropriate or Not-Safe-For-Work (NSFW) images. In this paper, inspired by the observation that texts with different semantics can lead to similar human perceptions, we propose an LLM-driven perception-guided jailbreak method, termed PGJ. It is a black-box jailbreak method that requires no specific T2I model (model-free) and generates highly natural attack prompts. Specifically, we propose identifying a safe phrase that is similar in human perception yet inconsistent in text semantics with the target unsafe word and using it as a substitution. The experiments conducted on six open-source models and commercial online services with thousands of prompts have verified the effectiveness of PGJ.

Key facts. Cited by 40 papers in the map (data through 14 Oct 2026). Placed in Vision-language and text-to-image safety, one of 12 directions in LLM safety and security. Builds on 36 of its 54 references that are in the map. First posted on arXiv 20 Aug 2024.

Builds on

36 of 54 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Jailbreaking Prompt Attack: A Controllable Adversarial Attack against Diffusion Models2 Apr
  2. 2023Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?16 Oct
  3. 2023SneakyPrompt: Jailbreaking Text-to-image Generative Models20 May
  4. 2022Red-Teaming the Stable Diffusion Safety Filter3 Oct

Other references · 32

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  6. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  7. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  8. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  9. 2022Diffusion Models in Vision: A Surveycited by 1,015
  10. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 860
  11. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  12. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 513
20 more
  1. 2023Text-to-image Diffusion Models in Generative AI: A Surveycited by 265
  2. 2023MMA-Diffusion: MultiModal Attack on Diffusion Modelscited by 194
  3. 2021Implicit Representations of Meaning in Neural Language Modelscited by 180
  4. 2023Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Modelscited by 175
  5. 2023Adversarial Example Does Good: Preventing Painting Imitation from Diffusion Models via Adversarial Examplescited by 167
  6. 2015Generating Images from Captions with Attentioncited by 136
  7. 2024Improved Techniques for Optimization-Based Jailbreaking on Large Language Modelscited by 110
  8. 2024BadEdit: Backdooring large language models by model editingcited by 96
  9. 2023A Pilot Study of Query-Free Adversarial Attack against Stable Diffusioncited by 76
  10. 2020Adv-watermark: A Novel Watermark Perturbation for Adversarial Examplescited by 56
  11. 2023Personalization as a Shortcut for Few-Shot Backdoor Attack against Text-to-Image Diffusion Modelscited by 42
  12. 2024A Vision Check-up for Language Modelscited by 36
  13. 2023Improving Fast Adversarial Training with Prior-Guided Knowledgecited by 35
  14. 2023SurrogatePrompt: Bypassing the Safety Filter of Text-to-Image Models via Substitutioncited by 34
  15. 2023Evaluating the Robustness of Text-to-image Diffusion Models against Real-world Attackscited by 24
  16. 2022A Tale of HodgeRank and Spectral Method: Target Attack Against Rank Aggregation Is the Fixed Point of Adversarial Gamecited by 16
  17. 2024On the Multi-modal Vulnerability of Diffusion Modelscited by 15
  18. 2024UPAM: Unified Prompt Attack in Text-to-Image Generation Models Against Both Textual Filters and Visual Checkerscited by 12
  19. 2022ALA: Naturalness-aware Adversarial Lightness Attackcited by 11
  20. 2024Texture Re-scalable Universal Adversarial Perturbationcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 18; 2026: 18.

40 papers.

Sort:

By area:

  1. 2025Adversarial Attacks against Closed-Source MLLMs via Feature Optimal Alignmentcited by 43
  2. 2024Heuristic-Induced Multimodal Risk Distribution Jailbreak Attack for Multimodal Large Language Modelscited by 33
  3. 2024Jailbreak Attacks and Defenses against Multimodal Generative Models: A Surveycited by 32
  4. 2025T2V-OptJail: Discrete Prompt Optimization for Text-to-Video Jailbreak Attackscited by 21
  5. 2025PromptGuard: Soft Prompt-Guided Unsafe Content Moderation for Text-to-Image Modelscited by 17
  6. 2025OmniSafeBench-MM: A Unified Benchmark and Toolbox for Multimodal Jailbreak Attack-Defense Evaluationcited by 11
  7. 2025Token-Level Constraint Boundary Search for Jailbreaking Text-to-Image Modelscited by 9
  8. 2024Understanding the Effectiveness of Coverage Criteria for Large Language Models: A Special Angle from Jailbreak Attackscited by 8
  9. 2025Red-Teaming Text-to-Image Systems by Rule-based Preference Modelingcited by 8
  10. 2025RunawayEvil: Jailbreaking the Image-to-Video Generative Modelscited by 8
  11. 2025Evolution-based Region Adversarial Prompt Learning for Robustness Enhancement in Vision-Language Modelscited by 6
  12. 2025AutoPrompt: Automated Red-Teaming of Text-to-Image Models via LLM-Driven Adversarial Promptscited by 6
28 more
  1. 2025When Memory Becomes a Vulnerability: Towards Multi-turn Jailbreak Attacks against Text-to-Image Generation Systemscited by 5
  2. 2025DREAM: Scalable Red Teaming for Text-to-Image Generative Systems via Distribution Modelingcited by 5
  3. 2026Low-Effort Jailbreak Attacks Against Text-to-Image Safety Filterscited by 5
  4. 2025GenBreak: Red Teaming Text-to-Image Generators Using Large Language Modelscited by 4
  5. 2025SPARK: Jailbreaking T2V Models by Synergistically Prompting Auditory and Recontextualized Knowledgecited by 4
  6. 2025Responsible Diffusion: A Comprehensive Survey on Safety, Ethics, and Trust in Diffusion Modelscited by 3
  7. 2026Conditioned Activation Transport for T2I Safety Steeringcited by 3
  8. 2026JANUS: A Lightweight Framework for Jailbreaking Text-to-Image Models via Distribution Optimizationcited by 3
  9. 2025Multimodal Prompt Decoupling Attack on the Safety Filters in Text-to-Image Modelscited by 2
  10. 2026Beyond Visual Safety: Jailbreaking Multimodal Large Language Models for Harmful Image Generation via Semantic-Agnostic Inputscited by 2
  11. 2025NDM: A Noise-driven Detection and Mitigation Framework against Implicit Sexual Intentions in Text-to-Image Generationcited by 1
  12. 2026Erased but Exploitable: Black-box Embedding-Aware Prompting Against Unlearned Text-to-Image Diffusion Modelscited by 1
  13. 2024An indicator for effectiveness of text-to-image guardrails utilizing the Single-Turn Crescendo Attack (STCA)cited by 0
  14. 2025CAHS-Attack: CLIP-Aware Heuristic Search Attack Method for Stable Diffusioncited by 0
  15. 2025Metaphor-based Jailbreak Attacks on Text-to-Image Modelscited by 0
  16. 2026FENCE: A Financial and Multimodal Jailbreak Detection Datasetcited by 0
  17. 2026When Safe Concepts Become Unsafe: Multi-Concept Compositional Vulnerabilities in Text-to-Image Modelscited by 0
  18. 2026STARE: Step-wise Temporal Alignment and Red-teaming Engine for Multi-modal Toxicity Attackcited by 0
  19. 2026OrchJail: Jailbreaking Tool-Calling Text-to-Image Agents by Orchestration-Guided Fuzzingcited by 0
  20. 2026PixJail: Self-Evolving Paper-to-Pipeline Reproduction for Text-to-Image Jailbreak Evaluationcited by 0
  21. 2026AEGIS: A Mechanism-Guided Defense against Visual Synonym Jailbreaks in Text-to-Image Modelscited by 0
  22. 2026UniNDM: A Unified Noise-driven Detection and Mitigation Framework Against Sexual Content in Text-to-Image Generationcited by 0
  23. 2026Attention from Above: A Multimodal Model for Drone-Based Object Localizationcited by 0
  24. 2026Multi2AV-Safety: Benchmarking Safety in Multimodal-to-Audio-Video Generationcited by 0
  25. 2026Jailbreaking Text-to-Image Models Through Cracks: Navigating Heterogeneous Safety Filters via Multi-Agent Debatecited by 0
  26. 2026RISE: Red-teaming via Iterative Strategy Evolution for Modern Text-to-Image Modelscited by 0
  27. 2026CollageAttack: Exploiting Cross-Modal Alignment Flaws in T2I Models through Spatial Text Compositioncited by 0
  28. 2026Safe Image Generation via Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Universally Unfiltered and Unseen:Input-Agnostic Multimodal Jailbreaks against Text-to-Image Model Safeguards
  2. 2025Anyone Can Jailbreak: Prompt-Based Attacks on LLMs and T2Is
  3. 2026Open Models, Open Risks: Measuring Unsafe Generation in Text-to-Image Models In the Wild
  4. 2024Multimodal Pragmatic Jailbreak on Text-to-image Models
  5. 2026Understanding and Defending VLM Jailbreaks via Jailbreak-Related Representation Shift
  6. 2024ImgTrojan: Jailbreaking Vision-Language Models with ONE Image
  7. 2025Behind the Mask: Benchmarking Camouflaged Jailbreaks in Large Language Models
  8. 2025Align is not Enough: Multimodal Universal Jailbreak Attack against Multimodal Large Language Models