Token-Level Constraint Boundary Search for Jailbreaking Text-to-Image Models
On the map: LLM safety and security → Vision-language and text-to-image safety
Abstract (arXiv)
Text-to-Image (T2I) generation has advanced rapidly in recent years, but they also raise safety concerns due to the potential production of harmful content. In the practical deployments, T2I services typically adopt full-chain defenses that combine a prompt checker, a securely trained generator, and a post-hoc image checker. Jailbreaking such full-chain systems is challenging in the black-box settings because prompt tokens form a discrete combinatorial space and the attack must satisfy multiple coupled constraints under sparse feedback and limited queries. To address these challenges, we propose Token-level Constraint Boundary Search (TCBS)-Attack, a novel query-based black-box jailbreak attack that searches for tokens located near the decision boundaries defined by text and image checkers. TCBS-Attack incorporates decision boundaries as constraint conditions to guide the evolutionary search of token populations, iteratively optimize tokens near these boundaries. Such evolutionary search process reduces the effective search space and improves query efficiency while preserving semantic coherence. Extensive experiments demonstrate that TCBS-Attack consistently outperforms state-of-the-art jailbreak attacks across various T2I models, including securely trained open-source models and commercial online services like DALL-E 3. TCBS-Attack achieves an ASR-4 of 52.5% and an ASR-1 of 22.0% on jailbreaking full-chain T2I models, significantly surpassing baseline methods.
Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in Vision-language and text-to-image safety, one of 12 directions in LLM safety and security. Builds on 39 of its 55 references that are in the map. First posted on arXiv 15 Apr 2025.
Builds on
39 of 55 references.
Closest ancestors · 10
Same direction, newest first
- 2024HTS-Attack: Heuristic Token Search for Jailbreaking Text-to-Image Models25 Aug
- 2024Perception-guided Jailbreak against Text-to-Image Models20 Aug
- 2024DiffZOO: A Purely Query-Based Black-Box Attack for Red-teaming Text-to-Image Generative Model via Zeroth Order Optimization18 Aug
- 2024Jailbreaking Prompt Attack: A Controllable Adversarial Attack against Diffusion Models2 Apr
- 2024GuardT2I: Defending Text-to-Image Models from Adversarial Prompts3 Mar
- 2023To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Now18 Oct
- 2023Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?16 Oct
- 2023FLIRT: Feedback Loop In-context Red Teaming8 Aug
- 2023SneakyPrompt: Jailbreaking Text-to-image Generative Models20 May
- 2023Black Box Adversarial Prompting for Foundation Models8 Feb
Other references · 29
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
- 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
- 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
- 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 785
- 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 671
17 more
- 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 560
- 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 513
- 2017Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Modelscited by 462
- 2020BAE: BERT-based Adversarial Examples for Text Classificationcited by 316
- 2023MMA-Diffusion: MultiModal Attack on Diffusion Modelscited by 194
- 2023Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Modelscited by 175
- 2023Adversarial Example Does Good: Preventing Painting Imitation from Diffusion Models via Adversarial Examplescited by 167
- 2023Raising the Cost of Malicious AI-Powered Image Editingcited by 163
- 2022Can Machines Help Us Answering Question 16 in Datasheets, and In Turn Reflecting on Inappropriate Content?cited by 155
- 2023A Pilot Study of Query-Free Adversarial Attack against Stable Diffusioncited by 76
- 2020Adv-watermark: A Novel Watermark Perturbation for Adversarial Examplescited by 56
- 2022"Real Attackers Don't Compute Gradients": Bridging the Gap Between Adversarial ML Research and Practicecited by 56
- 2023Personalization as a Shortcut for Few-Shot Backdoor Attack against Text-to-Image Diffusion Modelscited by 42
- 2023Evaluating the Robustness of Text-to-image Diffusion Models against Real-world Attackscited by 24
- 2024On the Multi-modal Vulnerability of Diffusion Modelscited by 15
- 2024Preventing Catastrophic Overfitting in Fast Adversarial Training: A Bi-level Optimization Perspectivecited by 13
- 2024Texture Re-scalable Universal Adversarial Perturbationcited by 9
Built on by
Citing papers per year: 2025: 5; 2026: 4.
9 papers.
- 2026SafeNeuron: Neuron-Level Safety Alignment for Large Language Modelscited by 10
- 2025Red-Teaming Text-to-Image Systems by Rule-based Preference Modelingcited by 8
- 2025Responsible Diffusion: A Comprehensive Survey on Safety, Ethics, and Trust in Diffusion Modelscited by 3
- 2025Your Mailbox Is Mine: Prompt Injection Attacks Against Real-World LLM Email Agentscited by 2
- 2025Enhancing the Effectiveness and Durability of Backdoor Attacks in Federated Learning through Maximizing Task Distinctioncited by 0
- 2025CAHS-Attack: CLIP-Aware Heuristic Search Attack Method for Stable Diffusioncited by 0
- 2026ReLAPSe: Reinforcement-Learning-trained Adversarial Prompt Search for Erased concepts in unlearned diffusion modelscited by 0
- 2026Concept Score Relearning: A Unified Cross-Architecture Attack on Concept Erasurecited by 0
- 2026RISE: Red-teaming via Iterative Strategy Evolution for Modern Text-to-Image Modelscited by 0
- 2026RISE: Red-teaming via Iterative Strategy Evolution for Modern Text-to-Image Models28 Sep 2026
- 2026Concept Score Relearning: A Unified Cross-Architecture Attack on Concept Erasure27 Sep 2026
- 2026SafeNeuron: Neuron-Level Safety Alignment for Large Language Models12 Feb 2026
- 2026ReLAPSe: Reinforcement-Learning-trained Adversarial Prompt Search for Erased concepts in unlearned diffusion models30 Jan 2026
- 2025CAHS-Attack: CLIP-Aware Heuristic Search Attack Method for Stable Diffusion26 Nov 2025
- 2025Responsible Diffusion: A Comprehensive Survey on Safety, Ethics, and Trust in Diffusion Models25 Sep 2025
- 2025Enhancing the Effectiveness and Durability of Backdoor Attacks in Federated Learning through Maximizing Task Distinction23 Sep 2025
- 2025Your Mailbox Is Mine: Prompt Injection Attacks Against Real-World LLM Email Agents3 Jul 2025
- 2025Red-Teaming Text-to-Image Systems by Rule-based Preference Modeling27 May 2025
Similar, not cited
Close in wording, no citation either way
- 2026JANUS: A Lightweight Framework for Jailbreaking Text-to-Image Models via Distribution Optimization
- 2026Jailbreaking Text-to-Image Models Through Cracks: Navigating Heterogeneous Safety Filters via Multi-Agent Debate
- 2026Between Safe Boundaries: Exploiting Temporal Consistency for Jailbreaking Text-To-Video Generation Models
- 2026Mind the Gap: Zero-Query Jailbreaks via Filter-Generator Discrepancy in Text-to-Image Systems
- 2026OrchJail: Jailbreaking Tool-Calling Text-to-Image Agents by Orchestration-Guided Fuzzing
- 2024Fuzz-Testing Meets LLM-Based Agents: An Automated and Efficient Framework for Jailbreaking Text-To-Image Generation Models
- 2026TempQ-Jail: Query-Constrained Candidate Ranking for Text-to-Video Jailbreak Attacks
- 2026PixJail: Self-Evolving Paper-to-Pipeline Reproduction for Text-to-Image Jailbreak Evaluation