To Think or Not to Think: Exploring the Unthinking Vulnerability in Large Reasoning Models
On the map: LLM reasoning → Reasoning for agents, planning and code
also in 1 more
Abstract (arXiv)
Large Reasoning Models (LRMs) are designed to solve complex tasks by generating explicit reasoning traces before producing final answers. However, we reveal a critical vulnerability in LRMs -- termed Unthinking Vulnerability -- wherein the thinking process can be bypassed by manipulating special delimiter tokens. It is empirically demonstrated to be widespread across mainstream LRMs, posing both a significant risk and potential utility, depending on how it is exploited. In this paper, we systematically investigate this vulnerability from both malicious and beneficial perspectives. On the malicious side, we introduce Breaking of Thought (BoT), a novel attack that enables adversaries to bypass the thinking process of LRMs, thereby compromising their reliability and availability. We present two variants of BoT: a training-based version that injects backdoor during the fine-tuning stage, and a training-free version based on adversarial attack during the inference stage. As a potential defense, we propose thinking recovery alignment to partially mitigate the vulnerability. On the beneficial side, we introduce Monitoring of Thought (MoT), a plug-and-play framework that allows model owners to enhance efficiency and safety. It is implemented by leveraging the same vulnerability to dynamically terminate redundant or risky reasoning through external monitoring. Extensive experiments show that BoT poses a significant threat to reasoning reliability, while MoT provides a practical solution for preventing overthinking and jailbreaking. Our findings expose an inherent flaw in current LRM architectures and underscore the need for more robust reasoning systems in the future.
Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 28 of its 59 references that are in the map. First posted on arXiv 16 Feb 2025.
Builds on
28 of 59 references.
Closest ancestors · 3
Same direction, newest first
- 2025OverThink: Slowdown Attacks on Reasoning LLMs4 Feb
- 2025o3-mini vs DeepSeek-R1: Which One is Safer?30 Jan
- 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code12 Mar
Other references · 25
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2024GPT-4o System Cardcited by 4,588
- 2023Let's Verify Step by Stepcited by 3,789
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
- 2025s1: Simple test-time scalingcited by 1,341
- 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMscited by 570
- 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 472
- 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 469
- 2024A StrongREJECT for Empty Jailbreakscited by 385
13 more
- 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 293
- 2020ONION: A Simple and Effective Defense Against Textual Backdoor Attackscited by 263
- 2024Token-Budget-Aware LLM Reasoningcited by 247
- 2024WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Modelscited by 223
- 2024AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMscited by 172
- 2024Marco-o1: Towards Open Reasoning Models for Open-Ended Solutionscited by 144
- 2024Multi-Step Reasoning with Large Language Models, a Surveycited by 143
- 2022BackdoorBench: A Comprehensive Benchmark of Backdoor Learningcited by 128
- 2024Improved Techniques for Optimization-Based Jailbreaking on Large Language Modelscited by 110
- 2023Enhancing Fine-Tuning Based Backdoor Defense with Sharpness-Aware Minimizationcited by 64
- 2024PoisonBench: Assessing Large Language Model Vulnerability to Data Poisoningcited by 33
- 2024ChatBug: A Common Vulnerability of Aligned LLMs Induced by Chat Templatescited by 30
- 2025Dynamic Chain-of-Thought: Towards Adaptive Deep Reasoningcited by 5
Built on by
Citing papers per year: 2025: 18; 2026: 4.
22 papers.
Sort:
By area:
- 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 384
- 2025Safety Tax: Safety Alignment Makes Your Large Reasoning Models Less Reasonablecited by 122
- 2025Safety in Large Reasoning Models: A Surveycited by 64
- 2025Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Modelscited by 37
- 2025Contextual Integrity in LLMs via Reasoning and Reinforcement Learningcited by 31
- 2025ShadowCoT: Cognitive Hijacking for Stealthy Reasoning Backdoors in LLMscited by 30
- 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 25
- 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Surveycited by 18
- 2025BadVideo: Stealthy Backdoor Attack against Text-to-Video Generationcited by 9
- 2025BadThink: Triggered Overthinking Attacks on Chain-of-Thought Reasoning in Large Language Modelscited by 8
- 2025BadReasoner: Planting Tunable Overthinking Backdoors into Large Reasoning Models for Fun or Profitcited by 7
- 2025Reasoning Introduces New Poisoning Attacks Yet Makes Them More Complicatedcited by 6
10 more
- 2025When Models Outthink Their Safety: Unveiling and Mitigating Self-Jailbreak in Large Reasoning Modelscited by 6
- 2025PEAR: Phase Entropy Aware Reward for Efficient Reasoningcited by 5
- 2025Thought Purity: A Defense Framework For Chain-of-Thought Attackcited by 4
- 2025AdvChain: Adversarial Chain-of-Thought Tuning for Robust Safety Alignment of Large Reasoning Modelscited by 4
- 2026Thought-Transfer: Indirect Targeted Poisoning Attacks on Chain-of-Thought Reasoning Modelscited by 3
- 2025Rethinking Reasoning: A Survey on Reasoning-based Backdoors in LLMscited by 2
- 2025RRTL: Red Teaming Reasoning Large Language Models in Tool Learningcited by 1
- 2026Reasoning as an Attack Surface: Adaptive Evolutionary CoT Jailbreaks for LLMscited by 0
- 2026Fool's Gold: Defensive Deception Against Safety-Removal Attacks on Open-Weight Modelscited by 0
- 2026Reasoning-Aware Compression: Identifying and Protecting Vulnerable Reasoning Circuits for Energy-Efficient LLM Deploymentcited by 0
- 2026Reasoning-Aware Compression: Identifying and Protecting Vulnerable Reasoning Circuits for Energy-Efficient LLM Deployment31 Aug 2026
- 2026Fool's Gold: Defensive Deception Against Safety-Removal Attacks on Open-Weight Models17 Aug 2026
- 2026Reasoning as an Attack Surface: Adaptive Evolutionary CoT Jailbreaks for LLMs23 May 2026
- 2026Thought-Transfer: Indirect Targeted Poisoning Attacks on Chain-of-Thought Reasoning Models27 Jan 2026
- 2025BadThink: Triggered Overthinking Attacks on Chain-of-Thought Reasoning in Large Language Models13 Nov 2025
- 2025When Models Outthink Their Safety: Unveiling and Mitigating Self-Jailbreak in Large Reasoning Models24 Oct 2025
- 2025PEAR: Phase Entropy Aware Reward for Efficient Reasoning9 Oct 2025
- 2025Rethinking Reasoning: A Survey on Reasoning-based Backdoors in LLMs9 Oct 2025
- 2025AdvChain: Adversarial Chain-of-Thought Tuning for Robust Safety Alignment of Large Reasoning Models29 Sep 2025
- 2025Reasoning Introduces New Poisoning Attacks Yet Makes Them More Complicated6 Sep 2025
- 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models4 Sep 2025
- 2025Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Models4 Aug 2025
10 more
- 2025BadReasoner: Planting Tunable Overthinking Backdoors into Large Reasoning Models for Fun or Profit24 Jul 2025
- 2025Thought Purity: A Defense Framework For Chain-of-Thought Attack16 Jul 2025
- 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey13 Jul 2025
- 2025Contextual Integrity in LLMs via Reasoning and Reinforcement Learning29 May 2025
- 2025RRTL: Red Teaming Reasoning Large Language Models in Tool Learning21 May 2025
- 2025Safety in Large Reasoning Models: A Survey24 Apr 2025
- 2025BadVideo: Stealthy Backdoor Attack against Text-to-Video Generation23 Apr 2025
- 2025ShadowCoT: Cognitive Hijacking for Stealthy Reasoning Backdoors in LLMs8 Apr 2025
- 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models12 Mar 2025
- 2025Safety Tax: Safety Alignment Makes Your Large Reasoning Models Less Reasonable1 Mar 2025
- 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 384
- 2025Safety Tax: Safety Alignment Makes Your Large Reasoning Models Less Reasonablecited by 122
- 2025Safety in Large Reasoning Models: A Surveycited by 64
- 2025Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Modelscited by 37
- 2025Contextual Integrity in LLMs via Reasoning and Reinforcement Learningcited by 31
- 2025ShadowCoT: Cognitive Hijacking for Stealthy Reasoning Backdoors in LLMscited by 30
- 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 25
- 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Surveycited by 18
- 2025BadThink: Triggered Overthinking Attacks on Chain-of-Thought Reasoning in Large Language Modelscited by 8
- 2025BadReasoner: Planting Tunable Overthinking Backdoors into Large Reasoning Models for Fun or Profitcited by 7
- 2025Reasoning Introduces New Poisoning Attacks Yet Makes Them More Complicatedcited by 6
- 2025When Models Outthink Their Safety: Unveiling and Mitigating Self-Jailbreak in Large Reasoning Modelscited by 6
12 most cited of 20 All 20 (JSON)
- 2026Reasoning-Aware Compression: Identifying and Protecting Vulnerable Reasoning Circuits for Energy-Efficient LLM Deployment31 Aug 2026
- 2026Reasoning as an Attack Surface: Adaptive Evolutionary CoT Jailbreaks for LLMs23 May 2026
- 2026Thought-Transfer: Indirect Targeted Poisoning Attacks on Chain-of-Thought Reasoning Models27 Jan 2026
- 2025BadThink: Triggered Overthinking Attacks on Chain-of-Thought Reasoning in Large Language Models13 Nov 2025
- 2025When Models Outthink Their Safety: Unveiling and Mitigating Self-Jailbreak in Large Reasoning Models24 Oct 2025
- 2025PEAR: Phase Entropy Aware Reward for Efficient Reasoning9 Oct 2025
- 2025Rethinking Reasoning: A Survey on Reasoning-based Backdoors in LLMs9 Oct 2025
- 2025AdvChain: Adversarial Chain-of-Thought Tuning for Robust Safety Alignment of Large Reasoning Models29 Sep 2025
- 2025Reasoning Introduces New Poisoning Attacks Yet Makes Them More Complicated6 Sep 2025
- 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models4 Sep 2025
- 2025Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Models4 Aug 2025
- 2025BadReasoner: Planting Tunable Overthinking Backdoors into Large Reasoning Models for Fun or Profit24 Jul 2025
12 newest of 20 All 20 (JSON)
Similar, not cited
Close in wording, no citation either way
- 2025Process or Result? Manipulated Ending Tokens Can Mislead Reasoning LLMs to Ignore the Correct Reasoning Steps
- 2026On the Efficiency-Safety Dilemma in Large Reasoning Models
- 2025Distractor Injection Attacks on Large Reasoning Models: Characterization and Defense
- 2026Reasoning Hijacking: The Fragility of Reasoning Alignment in Large Language Models
- 2025Think How to Think: Mitigating Overthinking with Autonomous Difficulty Cognition in Large Reasoning Models
- 2025H-CoT: Hijacking the Chain-of-Thought Safety Reasoning Mechanism to Jailbreak Large Reasoning Models, Including OpenAI o1/o3, DeepSeek-R1, and Gemini 2.0 Flash Thinking
- 2025One Token Embedding Is Enough to Deadlock Your Large Reasoning Model
- 2026Fragile Thoughts: How Large Language Models Handle Chain-of-Thought Perturbations