mlsanityData to 14 Oct 2026

Token-Level Constraint Boundary Search for Jailbreaking Text-to-Image Models

arXiv 2504.11106 ↗First posted 15 Apr 2025cs.CV cs.CRcited by 9

On the map: LLM safety and security → Vision-language and text-to-image safety

Abstract (arXiv)

Text-to-Image (T2I) generation has advanced rapidly in recent years, but they also raise safety concerns due to the potential production of harmful content. In the practical deployments, T2I services typically adopt full-chain defenses that combine a prompt checker, a securely trained generator, and a post-hoc image checker. Jailbreaking such full-chain systems is challenging in the black-box settings because prompt tokens form a discrete combinatorial space and the attack must satisfy multiple coupled constraints under sparse feedback and limited queries. To address these challenges, we propose Token-level Constraint Boundary Search (TCBS)-Attack, a novel query-based black-box jailbreak attack that searches for tokens located near the decision boundaries defined by text and image checkers. TCBS-Attack incorporates decision boundaries as constraint conditions to guide the evolutionary search of token populations, iteratively optimize tokens near these boundaries. Such evolutionary search process reduces the effective search space and improves query efficiency while preserving semantic coherence. Extensive experiments demonstrate that TCBS-Attack consistently outperforms state-of-the-art jailbreak attacks across various T2I models, including securely trained open-source models and commercial online services like DALL-E 3. TCBS-Attack achieves an ASR-4 of 52.5% and an ASR-1 of 22.0% on jailbreaking full-chain T2I models, significantly surpassing baseline methods.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in Vision-language and text-to-image safety, one of 12 directions in LLM safety and security. Builds on 39 of its 55 references that are in the map. First posted on arXiv 15 Apr 2025.

Builds on

39 of 55 references.

Closest ancestors · 10

Same direction, newest first

  1. 2024HTS-Attack: Heuristic Token Search for Jailbreaking Text-to-Image Models25 Aug
  2. 2024Perception-guided Jailbreak against Text-to-Image Models20 Aug
  3. 2024DiffZOO: A Purely Query-Based Black-Box Attack for Red-teaming Text-to-Image Generative Model via Zeroth Order Optimization18 Aug
  4. 2024Jailbreaking Prompt Attack: A Controllable Adversarial Attack against Diffusion Models2 Apr
  5. 2024GuardT2I: Defending Text-to-Image Models from Adversarial Prompts3 Mar
  6. 2023To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Now18 Oct
  7. 2023Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?16 Oct
  8. 2023FLIRT: Feedback Loop In-context Red Teaming8 Aug
  9. 2023SneakyPrompt: Jailbreaking Text-to-image Generative Models20 May
  10. 2023Black Box Adversarial Prompting for Foundation Models8 Feb

Other references · 29

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  8. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  9. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  10. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  11. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 785
  12. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 671
17 more
  1. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 560
  2. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 513
  3. 2017Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Modelscited by 462
  4. 2020BAE: BERT-based Adversarial Examples for Text Classificationcited by 316
  5. 2023MMA-Diffusion: MultiModal Attack on Diffusion Modelscited by 194
  6. 2023Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Modelscited by 175
  7. 2023Adversarial Example Does Good: Preventing Painting Imitation from Diffusion Models via Adversarial Examplescited by 167
  8. 2023Raising the Cost of Malicious AI-Powered Image Editingcited by 163
  9. 2022Can Machines Help Us Answering Question 16 in Datasheets, and In Turn Reflecting on Inappropriate Content?cited by 155
  10. 2023A Pilot Study of Query-Free Adversarial Attack against Stable Diffusioncited by 76
  11. 2020Adv-watermark: A Novel Watermark Perturbation for Adversarial Examplescited by 56
  12. 2022"Real Attackers Don't Compute Gradients": Bridging the Gap Between Adversarial ML Research and Practicecited by 56
  13. 2023Personalization as a Shortcut for Few-Shot Backdoor Attack against Text-to-Image Diffusion Modelscited by 42
  14. 2023Evaluating the Robustness of Text-to-image Diffusion Models against Real-world Attackscited by 24
  15. 2024On the Multi-modal Vulnerability of Diffusion Modelscited by 15
  16. 2024Preventing Catastrophic Overfitting in Fast Adversarial Training: A Bi-level Optimization Perspectivecited by 13
  17. 2024Texture Re-scalable Universal Adversarial Perturbationcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 4.

9 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026JANUS: A Lightweight Framework for Jailbreaking Text-to-Image Models via Distribution Optimization
  2. 2026Jailbreaking Text-to-Image Models Through Cracks: Navigating Heterogeneous Safety Filters via Multi-Agent Debate
  3. 2026Between Safe Boundaries: Exploiting Temporal Consistency for Jailbreaking Text-To-Video Generation Models
  4. 2026Mind the Gap: Zero-Query Jailbreaks via Filter-Generator Discrepancy in Text-to-Image Systems
  5. 2026OrchJail: Jailbreaking Tool-Calling Text-to-Image Agents by Orchestration-Guided Fuzzing
  6. 2024Fuzz-Testing Meets LLM-Based Agents: An Automated and Efficient Framework for Jailbreaking Text-To-Image Generation Models
  7. 2026TempQ-Jail: Query-Constrained Candidate Ranking for Text-to-Video Jailbreak Attacks
  8. 2026PixJail: Self-Evolving Paper-to-Pipeline Reproduction for Text-to-Image Jailbreak Evaluation