mlsanityData to 14 Oct 2026

Jailbreaking Text-to-Image Models Through Cracks: Navigating Heterogeneous Safety Filters via Multi-Agent Debate

arXiv 2609.01168 ↗First posted 1 Sep 2026cs.AI cs.MMcited by 0

On the map: LLM agents and tool use → Agent threats, guardrails and governance

also in 1 more
Abstract (arXiv)

Text-to-image (T2I) models remain vulnerable to jailbreak attacks that elicit Not-Safe-For-Work (NSFW) content, despite increasingly being guarded by heterogeneous, multi-layer safety stacks combining text filters, image classifiers, and cross-modal detectors. Existing jailbreak studies either optimize against individual filters or query the complete pipeline with aggregate feedback, making it difficult to identify the active constraint and adapt to conflicts across safety layers. In this paper, we introduce the Detection Surface, a unified geometric framework that characterizes the decision boundaries induced by heterogeneous T2I safety filters and their joint effect on the jailbreak search space. This formulation reveals that successful evasion is governed by a sparse and non-convex region shaped by cross-layer conflicts, where mutations that bypass one filter may increase exposure to another. Motivated by this analysis, we propose CRACK, a multi-agent debate framework for adaptive jailbreak search that decomposes jailbreak search into exploration, diagnosis, and arbitration. CRACK coordinates an Attack Agent, a Defense Agent, and a Judge Agent to iteratively generate prompt mutations, obtain layer-specific diagnostic feedback, and optimize mutation strategies through reward-guided refinement. Through repeated rounds of debate, CRACK adapts its search direction to the evolving cross-layer constraints while preserving the original harmful intent. Extensive experiments across multiple T2I models, datasets, and safety configurations show that CRACK achieves Attack Success Rates (ASR) of up to 99.63% under composite defenses, while requiring fewer queries than existing methods and maintaining semantic fidelity.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 36 of its 48 references that are in the map. First posted on arXiv 1 Sep 2026.

Builds on

36 of 48 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Fuzz-Testing Meets LLM-Based Agents: An Automated and Efficient Framework for Jailbreaking Text-To-Image Generation Models1 Aug

Other references · 35

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2017Proximal Policy Optimization Algorithmscited by 15,215
  5. 2025Qwen3 Technical Reportcited by 8,820
  6. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  7. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  8. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  9. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  10. 2022Red Teaming Language Models with Language Modelscited by 1,026
  11. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 1,011
  12. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 672
23 more
  1. 2023Erasing Concepts from Diffusion Modelscited by 539
  2. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 513
  3. 2014On the Number of Linear Regions of Deep Neural Networkscited by 381
  4. 2022Red-Teaming the Stable Diffusion Safety Filtercited by 264
  5. 2023Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?cited by 250
  6. 2024Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Promptscited by 205
  7. 2023MMA-Diffusion: MultiModal Attack on Diffusion Modelscited by 194
  8. 2023SneakyPrompt: Jailbreaking Text-to-image Generative Modelscited by 189
  9. 2023Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Modelscited by 175
  10. 2023Prompting4Debugging: Red-Teaming Text-to-Image Diffusion Models by Finding Problematic Promptscited by 171
  11. 2022Can Machines Help Us Answering Question 16 in Datasheets, and In Turn Reflecting on Inappropriate Content?cited by 155
  12. 2023Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMscited by 114
  13. 2020Improving Image Captioning with Better Use of Captionscited by 84
  14. 2024Controllable Generation with Text-to-Image Diffusion Models: A Surveycited by 83
  15. 2020Actor-Critic Reinforcement Learning for Control with Stability Guaranteecited by 45
  16. 2025AutoRedTeamer: Autonomous Red Teaming with Lifelong Attack Integrationcited by 43
  17. 2024Perception-guided Jailbreak against Text-to-Image Modelscited by 40
  18. 2023DreamStyler: Paint by Style Inversion with Text-to-Image Diffusion Modelscited by 27
  19. 2024Synergistic Multi-Agent Framework with Trajectory Learning for Knowledge-Intensive Taskscited by 23
  20. 2023Harnessing LLM to Attack LLM-Guarded Text-to-Image Modelscited by 12
  21. 2025Dynamic Optimization and Safety Indicator Injection for Jailbreaking Text-to-Image Models with Multimodal Safety Filterscited by 5
  22. 2026JANUS: A Lightweight Framework for Jailbreaking Text-to-Image Models via Distribution Optimizationcited by 3
  23. 2026MacPrompt: Maraconic-guided Jailbreak against Text-to-Image Modelscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Token-Level Constraint Boundary Search for Jailbreaking Text-to-Image Models
  2. 2026OrchJail: Jailbreaking Tool-Calling Text-to-Image Agents by Orchestration-Guided Fuzzing
  3. 2026Mind the Gap: Zero-Query Jailbreaks via Filter-Generator Discrepancy in Text-to-Image Systems
  4. 2026Low-Effort Jailbreak Attacks Against Text-to-Image Safety Filters
  5. 2025DefenSee: Dissecting Threat from Sight and Text -- A Multi-View Defensive Pipeline for Multi-modal Jailbreaks
  6. 2025JailDAM: Jailbreak Detection with Adaptive Memory for Vision-Language Model
  7. 2025Universally Unfiltered and Unseen:Input-Agnostic Multimodal Jailbreaks against Text-to-Image Model Safeguards
  8. 2026Open Models, Open Risks: Measuring Unsafe Generation in Text-to-Image Models In the Wild