mlsanityData to 7 Oct 2026

Chain-of-Thought Hijacking

arXiv 2510.26418 ↗First posted 30 Oct 2025cs.AIcited by 13

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

Large Reasoning Models (LRMs) improve task performance through extended inference-time reasoning. Although previous studies suggest that longer reasoning should lead to more robust safety behavior, we find evidence to the contrary: over-extended reasoning can instead be exploited to systematically weaken refusal behavior. We propose Chain-of-Thought Hijacking, a simple yet effective black-box jailbreak attack that induces LRMs to engage in prolonged benign puzzle-solving reasoning, often lasting more than five minutes, before eliciting harmful compliance. Across HarmBench, CoT Hijacking achieves attack success rates of 99%, 94%, 100%, and 94% on Gemini 2.5 Pro, ChatGPT o4 Mini, Grok 3 Mini, and Claude 4 Sonnet, respectively. To understand why this attack succeeds, we conduct activation probing, attention-pattern analysis, and causal interventions on open-source reasoning models. Our results indicate that refusal behavior depends on a low-dimensional safety signal whose expression weakens as reasoning traces grow longer. In particular, extended benign reasoning shifts attention away from harmful intentions and attenuates refusal-related activations, producing what we call refusal dilution. These findings demonstrate that excessively prolonged reasoning can introduce a systematic jailbreak attack surface. We release our evaluation materials to support reproducibility and further research.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 35 of its 54 references that are in the map. First posted on arXiv 30 Oct 2025.

Builds on

35 of 54 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025AutoRAN: Automated Hijacking of Safety Reasoning in Large Reasoning Models16 May
  2. 2025A Mousetrap: Fooling Large Reasoning Models for Jailbreak with Chain of Iterative Chaos19 Feb
  3. 2025H-CoT: Hijacking the Chain-of-Thought Safety Reasoning Mechanism to Jailbreak Large Reasoning Models, Including OpenAI o1/o3, DeepSeek-R1, and Gemini 2.0 Flash Thinking18 Feb
  4. 2025The Hidden Risks of Large Reasoning Models: A Safety Assessment of R118 Feb
  5. 2024OpenAI o1 System Card21 Dec
  6. 2024Deliberative Alignment: Reasoning Enables Safer Language Models20 Dec

Other references · 29

  1. 2025Qwen3 Technical Reportcited by 8,195
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  4. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  5. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  6. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
  7. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 988
  8. 2025Kimi K2: Open Agentic Intelligencecited by 523
  9. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Ciphercited by 401
  10. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 337
  11. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  12. 2024COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllabilitycited by 162
17 more
  1. 2025Seed1.5-Thinking: Advancing Superb Reasoning Models with Reinforcement Learningcited by 159
  2. 2024Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Modelscited by 158
  3. 2024The Language Barrier: Dissecting Safety Challenges of LLMs in Multilingual Contextscited by 92
  4. 2025Concise Reasoning via Reinforcement Learningcited by 78
  5. 2024Best-of-N Jailbreakingcited by 71
  6. 2024On the Role of Attention Heads in Large Language Model Safetycited by 70
  7. 2024Visual-RolePlay: Universal Jailbreak Attack on MultiModal Large Language Models via Role-playing Image Charactercited by 61
  8. 2024Towards Understanding Jailbreak Attacks in LLMs: A Representation Space Analysiscited by 61
  9. 2025When Thinking Fails: The Pitfalls of Reasoning for Instruction-Following in LLMscited by 51
  10. 2024Towards Understanding Safety Alignment: A Mechanistic Perspective from Safety Neuronscited by 47
  11. 2024Understanding Jailbreak Success: A Study of Latent Space Dynamics in Large Language Modelscited by 34
  12. 2024PoisonBench: Assessing Large Language Model Vulnerability to Data Poisoningcited by 31
  13. 2025Scaling Reasoning, Losing Control: Evaluating Instruction Following in Large Reasoning Modelscited by 28
  14. 2024JailbreakLens: Interpreting Jailbreak Mechanism in the Lens of Representation and Circuitcited by 21
  15. 2025SafeSteer: Interpretable Safety Steering with Refusal-Evasion in LLMscited by 9
  16. 2024Jailbreak Defense in a Narrow Domain: Limitations of Existing Methods and a New Transcript-Classifier Approachcited by 6
  17. 2025Beyond Black-Box Obfuscation: Mechanistic Analysis and Defense of White-Box Monitorscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 13.

13 papers.

Sort:

By area:

  1. 2026A Systematic Survey of Security Threats and Defenses in LLM-Based AI Agents: A Layered Attack Surface Frameworkcited by 9
  2. 2026Do Reasoning LLMs Refuse What They Infer in Long Contexts?cited by 3
  3. 2026The Side Effects of Being Smart: Safety Risks in MLLMs' Multi-Image Reasoningcited by 2
  4. 2026Towards Safer Large Reasoning Models by Promoting Safety Decision-Making before Chain-of-Thought Generationcited by 2
  5. 2026Jailbreaking LLMs via Calibrationcited by 0
  6. 2026Attention-Guided Reward for Reinforcement Learning-based Jailbreak against Large Reasoning Modelscited by 0
  7. 2026StructBreak: Structural Cognitive Overload-Induced Safety Failures in MLLMscited by 0
  8. 2026Beyond a Single Direction: Chain-of-Thought Disrupts Simple Steering of Refusalcited by 0
  9. 2026Mitigating Adaptive Attacks against Reasoning Models with Activation Consistency Trainingcited by 0
  10. 2026First Token Matters: Understanding Safety Collapse in Large Reasoning Modelscited by 0
  11. 2026Taming CoT Obfuscation in VLMs: From Mechanistic Evidence to Activation Enforcementcited by 0
  12. 2026Alignment of LRMs via Counter-Aligned Few-Shot Conversation Exposurecited by 0
1 more
  1. 2026Towards Mitigating Deceptive Safety Alignment in Large Reasoning Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Refusal Falls off a Cliff: How Safety Alignment Fails in Reasoning?
  2. 2025BadThink: Triggered Overthinking Attacks on Chain-of-Thought Reasoning in Large Language Models
  3. 2026Thought-Transfer: Indirect Targeted Poisoning Attacks on Chain-of-Thought Reasoning Models
  4. 2025Thought Purity: A Defense Framework For Chain-of-Thought Attack
  5. 2025When Models Outthink Their Safety: Unveiling and Mitigating Self-Jailbreak in Large Reasoning Models
  6. 2026Evading Chain-of-Thought Monitoring Through Model Poisoning
  7. 2025ShadowCoT: Cognitive Hijacking for Stealthy Reasoning Backdoors in LLMs
  8. 2025Chain-of-Defensive-Thought: Structured Reasoning Elicits Robustness in Large Language Models against Reference Corruption