mlsanityData to 7 Oct 2026

DoubtProbe: Black-Box Jailbreak Defense via Structural Verification and Semantic Auditing

arXiv 2606.16527 ↗First posted 15 Jun 2026cs.CR cs.CLcited by 0

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

As large language models (LLMs) are increasingly deployed in user-facing systems, black-box jailbreak defense has become an important practical problem. Existing defenses often rely on known-attack coverage, prompt-level semantic judgment, or local runtime control, yet these paths can become unstable under evolving prompt packaging, expression rewriting, and structure manipulation. We observe that many black-box jailbreaks do not remove the harmful goal, but reorganize the information needed to express and execute it, thereby evading safety alignment while remaining recoverable during generation. Motivated by this observation, we propose DoubtProbe, a dual-branch inference-time defense framework that combines structural verification with semantic auditing and formulates black-box jailbreak defense as consistency checking under controlled transformation. The structural branch extracts a structured representation from the original request, reconstructs the request under representation constraints, and detects information-preservation failures between the original and reconstructed requests; the semantic branch audits the original prompt directly. We evaluate DoubtProbe against representative black-box defenses on jailbreak and benign-request benchmarks, and further test backbone transfer from Qwen2.5-72B to Llama-3.1-70B. Results show that DoubtProbe achieves a stronger and more stable defense-utility trade-off: on Qwen2.5-72B, it reduces the JBB attack success rate from 0.293 to 0.100 and the CodeAttack attack success rate from 0.152 to 0.001, while maintaining false positive rates of 0.022 and 0.016 on AlpacaEval and OR-Bench; the same pattern remains stable on Llama-3.1-70B. These findings show that structural inconsistency signals provide a practical and generalizable basis for black-box jailbreak defense, especially when combined with semantic auditing.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 43 of its 58 references that are in the map. First posted on arXiv 15 Jun 2026.

Builds on

43 of 58 references.

Closest ancestors · 23

Same direction, newest first

  1. 2025Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming31 Jan
  2. 2024RePD: Defending Jailbreak Attack through a Retrieval-based Prompt Decomposition Process11 Oct
  3. 2024RoleBreak: Character Hallucination as a Jailbreak Attack in Role-Playing Systems25 Sep
  4. 2024SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Manner8 Jun
  5. 2024Don't Say No: Jailbreaking LLM by Suppressing Refusal25 Apr
  6. 2024AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMs11 Apr
  7. 2024JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks3 Apr
  8. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks2 Apr
  9. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack2 Apr
  10. 2024Defending Large Language Models against Jailbreak Attacks via Semantic Smoothing25 Feb
  11. 2024Play Guessing Game with LLM: Indirect Jailbreak Attack with Implicit Clues14 Feb
  12. 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decoding14 Feb
11 more
  1. 2024Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks30 Jan
  2. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs12 Jan
  3. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automatically4 Dec
  4. 2023Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritization15 Nov
  5. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreaker6 Nov
  6. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  7. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations10 Oct
  8. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks5 Oct
  9. 2023Certifying LLM Safety against Adversarial Prompting6 Sep
  10. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  11. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul

Other references · 20

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  3. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  4. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 477
  5. 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 460
  6. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Modelscited by 307
  7. 2023Poisoning Language Models During Instruction Tuningcited by 254
  8. 2023LLM Self Defense: By Self Examination, LLMs Know They Are Being Trickedcited by 226
  9. 2023ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversationcited by 224
  10. 2024ShieldGemma: Generative AI Content Moderation Based on Gemmacited by 224
  11. 2024OR-Bench: An Over-Refusal Benchmark for Large Language Modelscited by 222
  12. 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMscited by 197
8 more
  1. 2024SecAlign: Defending Against Prompt Injection with Preference Optimizationcited by 179
  2. 2023RAIN: Your Language Models Can Align Themselves without Finetuningcited by 149
  3. 2024AEGIS: Online Adaptive AI Content Safety Moderation with Ensemble of LLM Expertscited by 126
  4. 2024Optimization-based Prompt Injection Attack to LLM-as-a-Judgecited by 123
  5. 2024CodeAttack: Revealing Safety Generalization Challenges of Large Language Models via Code Completioncited by 70
  6. 2023The Art of Defending: A Systematic Evaluation and Analysis of LLM Defense Strategies on Safety and Over-Defensivenesscited by 63
  7. 2023Bergeron: Combating Adversarial Attacks through a Conscience-Based Alignment Frameworkcited by 29
  8. 2025AgentSentinel: An End-to-End and Real-Time Security Defense Framework for Computer-Use Agentscited by 25

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026JailMeter: An Evidence-Based Evaluation Framework for Jailbreak Attacks on Large Language Models
  2. 2026TRYLOCK: Defense-in-Depth Against LLM Jailbreaks via Layered Preference and Representation Engineering
  3. 2026Beyond the Prompt: Jailbreaking Function-Calling LLMs via Simulated Moderation Traces
  4. 2025Logic Jailbreak: Efficiently Unlocking LLM Safety Restrictions Through Formal Logical Expression
  5. 2024LIAR: Leveraging Inference Time Alignment (Best-of-N) to Jailbreak LLMs in Seconds
  6. 2025Retrieval-Augmented Defense: Adaptive and Controllable Jailbreak Prevention for Large Language Models
  7. 2025JBShield: Defending Large Language Models from Jailbreak Attacks through Activated Concept Analysis and Manipulation
  8. 2024BlackDAN: A Black-Box Multi-Objective Approach for Effective and Contextual Jailbreaking of Large Language Models