mlsanityData to 14 Oct 2026

Involuntary Jailbreak: On Self-Prompting Attacks

arXiv 2508.13246 ↗First posted 18 Aug 2025cs.CR cs.AIcited by 0

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

In this study, we disclose a worrying new vulnerability in Large Language Models (LLMs), which we term involuntary jailbreak. Unlike existing jailbreak attacks, this weakness is distinct in that it does not involve a specific attack objective, such as generating instructions for textitbuilding a bomb. Prior attack methods predominantly target localized components of the LLM guardrail. In contrast, involuntary jailbreaks may potentially compromise the entire guardrail structure, which our method reveals to be surprisingly fragile. We merely employ a single universal prompt to achieve this goal. In particular, we instruct LLMs to generate several questions that would typically be rejected, along with their corresponding in-depth responses (rather than a refusal). Remarkably, this simple prompt strategy consistently jailbreaks the majority of leading LLMs, including Claude Opus 4.1, Grok 4, Gemini 2.5 Pro, and GPT 4.1. We hope this problem can motivate researchers and practitioners to re-evaluate the robustness of LLM guardrails and contribute to stronger safety alignment in future.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 33 of its 41 references that are in the map. First posted on arXiv 18 Aug 2025.

Builds on

33 of 41 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025from Benign import Toxic: Jailbreaking the Language Model via Adversarial Metaphors25 Feb
  2. 2025Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming31 Jan
  3. 2024Large Language Models Are Involuntary Truth-Tellers: Exploiting Fallacy Failure for Jailbreak Attacks1 Jul
  4. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks2 Apr
  5. 2024A StrongREJECT for Empty Jailbreaks15 Feb
  6. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation10 Oct
  7. 2023Multilingual Jailbreak Challenges in Large Language Models10 Oct
  8. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models3 Oct
  9. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher12 Aug
  10. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  11. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul

Other references · 22

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  6. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  7. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  8. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  9. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  10. 2023LIMA: Less Is More for Alignmentcited by 1,100
  11. 2019Machine Unlearningcited by 1,061
  12. 2022Red Teaming Language Models with Language Modelscited by 1,026
10 more
  1. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 729
  2. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 472
  3. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Promptscited by 398
  4. 2024Alignment faking in large language modelscited by 360
  5. 2025Demystifying Long Chain-of-Thought Reasoning in LLMscited by 326
  6. 2024ShieldGemma: Generative AI Content Moderation Based on Gemmacited by 243
  7. 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusalcited by 240
  8. 2023Removing RLHF Protections in GPT-4 via Fine-Tuningcited by 171
  9. 2025Safety Tax: Safety Alignment Makes Your Large Reasoning Models Less Reasonablecited by 122
  10. 2024Does Refusal Training in LLMs Generalize to the Past Tense?cited by 79

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily
  2. 2025SoK: Evaluating Jailbreak Guardrails for Large Language Models
  3. 2026Re-Triggering Safeguards within LLMs for Jailbreak Detection
  4. 2024Bag of Tricks: Benchmarking of Jailbreak Attacks on LLMs
  5. 2025Do Internal Layers of LLMs Reveal Patterns for Jailbreak Detection?
  6. 2026Break Me If You Can: Self-Jailbreaking of Aligned LLMs via Lexical Insertion Prompting
  7. 2026Analysis of LLMs Against Prompt Injection and Jailbreak Attacks
  8. 2024Defensive Prompt Patch: A Robust and Interpretable Defense of LLMs against Jailbreak Attacks