mlsanityData to 7 Oct 2026

H-CoT: Hijacking the Chain-of-Thought Safety Reasoning Mechanism to Jailbreak Large Reasoning Models, Including OpenAI o1/o3, DeepSeek-R1, and Gemini 2.0 Flash Thinking

arXiv 2502.12893 ↗First posted 18 Feb 2025cs.CLcited by 94

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

Large Reasoning Models (LRMs) have recently extended their powerful reasoning capabilities to safety checks-using chain-of-thought reasoning to decide whether a request should be answered. While this new approach offers a promising route for balancing model utility and safety, its robustness remains underexplored. To address this gap, we introduce Malicious-Educator, a benchmark that disguises extremely dangerous or malicious requests beneath seemingly legitimate educational prompts. Our experiments reveal severe security flaws in popular commercial-grade LRMs, including OpenAI o1/o3, DeepSeek-R1, and Gemini 2.0 Flash Thinking. For instance, although OpenAI's o1 model initially maintains a high refusal rate of about 98%, subsequent model updates significantly compromise its safety; and attackers can easily extract criminal strategies from DeepSeek-R1 and Gemini 2.0 Flash Thinking without any additional tricks. To further highlight these vulnerabilities, we propose Hijacking Chain-of-Thought (H-CoT), a universal and transferable attack method that leverages the model's own displayed intermediate reasoning to jailbreak its safety reasoning mechanism. Under H-CoT, refusal rates sharply decline-dropping from 98% to below 2%-and, in some instances, even transform initially cautious tones into ones that are willing to provide harmful content. We hope these findings underscore the urgent need for more robust safety mechanisms to preserve the benefits of advanced reasoning capabilities without compromising ethical standards.

Key facts. Cited by 94 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 30 of its 40 references that are in the map. First posted on arXiv 18 Feb 2025.

Builds on

30 of 40 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Deliberative Alignment: Reasoning Enables Safer Language Models20 Dec
  2. 2024R2-Guard: Robust Reasoning Enabled LLM Guardrail via Knowledge-Enhanced Logical Reasoning8 Jul

Other references · 28

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2023Visual Instruction Tuningcited by 7,792
  6. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  7. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  8. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  9. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  10. 2023PaLM 2 Technical Reportcited by 1,154
  11. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  12. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
16 more
  1. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Ciphercited by 401
  2. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 401
  3. 2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Searchcited by 338
  4. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 337
  5. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 215
  6. 2023Towards Building the Federated GPT: Federated Instruction Tuningcited by 184
  7. 2024Min-K%++: Improved Baseline for Detecting Pre-Training Data from Large Language Modelscited by 102
  8. 2024Weak-to-Strong Jailbreaking on Large Language Modelscited by 100
  9. 2024Making Them Ask and Answer: Jailbreaking Large Language Models in Few Queries via Disguise and Reconstructioncited by 95
  10. 2024LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shiftscited by 80
  11. 2024Sufficient Context: A New Lens on Retrieval Augmented Generation Systemscited by 69
  12. 2024The Benefits of a Concise Chain of Thought on Problem-Solving in Large Language Modelscited by 62
  13. 2024Jailbreaking and Mitigation of Vulnerabilities in Large Language Modelscited by 33
  14. 2024Perceive, Reflect, and Plan: Designing LLM Agent for Goal-Directed City Navigation without Instructionscited by 25
  15. 2024Federated Large Language Models: Current Progress and Future Directionscited by 22
  16. 2024SLED: Self Logits Evolution Decoding for Improving Factuality in Large Language Modelscited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 59; 2026: 35.

94 papers.

Sort:

By area:

  1. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 456
  2. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  3. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  4. 2025Efficient Reasoning Models: A Surveycited by 93
  5. 2025Safety in Large Reasoning Models: A Surveycited by 58
  6. 2025Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Modelscited by 36
  7. 2025Contextual Integrity in LLMs via Reasoning and Reinforcement Learningcited by 30
  8. 2025100 Days After DeepSeek-R1: A Survey on Replication Studies and More Directions for Reasoning Language Modelscited by 29
  9. 2025Cats Confuse Reasoning LLM: Query Agnostic Adversarial Triggers for Reasoning Modelscited by 23
  10. 2025Visual Contextual Attack: Jailbreaking MLLMs with Image-Driven Context Injectioncited by 23
  11. 2025Reasoning as an Adaptive Defense for Safetycited by 22
  12. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 22
38 more
  1. 2025Trade-offs in Large Reasoning Models: An Empirical Analysis of Deliberative and Adaptive Reasoning over Foundational Capabilitiescited by 20
  2. 2025SafeKey: Amplifying Aha-Moment Insights for Safety Reasoningcited by 20
  3. 2025Steering the CensorShip: Uncovering Representation Vectors for LLM "Thought" Controlcited by 14
  4. 2025SAFEx: Analyzing Vulnerabilities of MoE-Based LLMs via Stable Safety-critical Expert Identificationcited by 14
  5. 2025dVLM-AD: Enhance Diffusion Vision-Language-Model for Driving via Controllable Reasoningcited by 14
  6. 2025Angles Don't Lie: Unlocking Training-Efficient RL Through the Model's Own Signalscited by 13
  7. 2025Chain-of-Thought Hijackingcited by 13
  8. 2025Towards Safe Reasoning in Large Reasoning Models via Corrective Interventioncited by 11
  9. 2025Refusal Falls off a Cliff: How Safety Alignment Fails in Reasoning?cited by 11
  10. 2025A Survey of Scaling in Large Language Model Reasoningcited by 10
  11. 2025Response Attack: Exploiting Contextual Priming to Jailbreak Large Language Modelscited by 10
  12. 2026Obscure but Effective: Classical Chinese Jailbreak Prompt Optimization via Bio-Inspired Searchcited by 10
  13. 2025ReasoningShield: Safety Detection over Reasoning Traces of Large Reasoning Modelscited by 9
  14. 2025Reasoning or Retrieval? A Study of Answer Attribution on Large Reasoning Modelscited by 9
  15. 2025Exposing LLM User Privacy via Traffic Fingerprint Analysis: A Study of Privacy Risks in LLM Agent Interactionscited by 9
  16. 2025Self-Jailbreaking: Language Models Can Reason Themselves Out of Safety Alignment After Benign Reasoning Trainingcited by 9
  17. 2025CoreMatching: A Co-adaptive Sparse Inference Framework with Token and Neuron Pruning for Comprehensive Acceleration of Vision-Language Modelscited by 8
  18. 2025Practical Reasoning Interruption Attacks on Reasoning Large Language Modelscited by 7
  19. 2025AutoRAN: Automated Hijacking of Safety Reasoning in Large Reasoning Modelscited by 7
  20. 2025Beyond Safe Answers: A Benchmark for Evaluating True Risk Awareness in Large Reasoning Modelscited by 7
  21. 2025SafeTy Reasoning Elicitation Alignment for Multi-Turn Dialoguescited by 7
  22. 2025LaSM: Layer-wise Scaling Mechanism for Defending Pop-up Attack on GUI Agentscited by 6
  23. 2025Adversarial D'ej`a Vu: Jailbreak Dictionary Learning for Stronger Generalization to Unseen Attackscited by 6
  24. 2026What Matters For Safety Alignment?cited by 6
  25. 2026Knowledge-Driven Multi-Turn Jailbreaking on Large Language Modelscited by 6
  26. 2025PEAR: Phase Entropy Aware Reward for Efficient Reasoningcited by 5
  27. 2025Path Drift in Large Reasoning Models:How First-Person Commitments Override Safetycited by 5
  28. 2026Towards a Mechanistic Understanding of Large Reasoning Models: A Survey of Training, Inference, and Failurescited by 5
  29. 2026TRAP: Hijacking VLA CoT-Reasoning via Adversarial Patchescited by 5
  30. 2025Reasoning Can Hurt the Inductive Abilities of Large Language Modelscited by 4
  31. 2025The Other Mind: How Language Models Exhibit Human Temporal Cognitioncited by 4
  32. 2025AdvChain: Adversarial Chain-of-Thought Tuning for Robust Safety Alignment of Large Reasoning Modelscited by 4
  33. 2025GuardTrace-VL: Detecting Unsafe Multimodel Reasoning via Iterative Safety Supervisioncited by 4
  34. 2025Where Do Reasoning Models Refuse?cited by 3
  35. 2026How Does the Thinking Step Influence Model Safety? An Entropy-based Safety Reminder for LRMscited by 3
  36. 2026The Echo Chamber Multi-Turn LLM Jailbreakcited by 3
  37. 2026Do Reasoning LLMs Refuse What They Infer in Long Contexts?cited by 3
  38. 2026Internalizing Safety Understanding in Large Reasoning Models via Verificationcited by 3

50 most cited of 94 All 94 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025SAFEPATH: Preventing Harmful Reasoning in Chain-of-Thought via Early Alignment
  2. 2025Thought Purity: A Defense Framework For Chain-of-Thought Attack
  3. 2025UnsafeChain: Enhancing Reasoning Model Safety via Hard Cases
  4. 2025SafeChain: Safety of Language Models with Long Chain-of-Thought Reasoning Capabilities
  5. 2026Beyond Content Safety: Real-Time Monitoring for Reasoning Vulnerabilities in Large Language Models
  6. 2025The dark deep side of DeepSeek: Fine-tuning attacks against the safety alignment of CoT-enabled models
  7. 2025When Models Outthink Their Safety: Unveiling and Mitigating Self-Jailbreak in Large Reasoning Models
  8. 2025The Hidden Risks of Large Reasoning Models: A Safety Assessment of R1