mlsanityData to 14 Oct 2026

When Can Large Reasoning Models Save Thinking? Mechanistic Analysis of Behavioral Divergence in Reasoning

arXiv 2505.15276 ↗First posted 21 May 2025cs.AI cs.CLcited by 7

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Large reasoning models (LRMs) have achieved remarkable success on complex tasks, yet their tendency to "overthink" leads to inefficiencies. Although "save-thinking" prompts are intended to mitigate this issue, we find that LRMs still frequently enter the "Still-thinking" mode instead of the expected "No-thinking" mode, especially on difficult queries. To analyze this behavioral divergence, we examine LRMs from three perspectives: confidence at the thinking-termination boundary, divergence in internal attention distributions, and attention allocation across prompt segments. We find that high perplexity is associated with later Still-thinking behavior, and that Still-thinking cases allocate more attention to the original question. Based on these observations, we propose an attention intervention method to regulate this behavior. While this intervention suppresses explicit thinking, it also causes a drop in accuracy, suggesting that the suppressed reasoning behavior is often useful for correctness. Our work provides confidence- and attention-level evidence for this behavior, highlighting the trade-off between instruction following, inference efficiency, and reasoning correctness.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 19 of its 33 references that are in the map. First posted on arXiv 21 May 2025.

Builds on

19 of 33 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Dynamic Early Exit in Reasoning Models22 Apr
  2. 2025Reasoning Models Can Be Effective Without Thinking14 Apr
  3. 2025Thinking Out Loud: Do Reasoning Models Know When They're Right?9 Apr
  4. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models20 Mar
  5. 2025s1: Simple test-time scaling31 Jan
  6. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs30 Dec
  7. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug

Other references · 12

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  4. 2024Qwen2.5 Technical Reportcited by 4,942
  5. 2023A Survey of Large Language Modelscited by 2,576
  6. 2024DeepSeek-V3 Technical Reportcited by 811
  7. 2019GLTR: Statistical Detection and Visualization of Generated Textcited by 496
  8. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 384
  9. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 293
  10. 2024The Ultimate Guide to Fine-Tuning LLMs from Basics to Breakthroughs: An Exhaustive Review of Technologies, Research, Best Practices, Applied Research Challenges and Opportunitiescited by 64
  11. 2025Mitigating Lost-in-Retrieval Problems in Retrieval Augmented Multi-Hop Question Answeringcited by 22
  12. 2024Mathify: Evaluating Large Language Models on Mathematical Problem Solving Taskscited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 4.

7 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026To Think or Not To Think, That is The Question for Large Reasoning Models in Theory of Mind Tasks
  2. 2025Large Reasoning Models are not thinking straight: on the unreliability of thinking trajectories
  3. 2025Measuring the Faithfulness of Thinking Drafts in Large Reasoning Models
  4. 2025Longer Context, Deeper Thinking: Uncovering the Role of Long-Context Ability in Reasoning
  5. 2025Reasoning Strategies in Large Language Models: Can They Follow, Prefer, and Optimize?
  6. 2025Think How to Think: Mitigating Overthinking with Autonomous Difficulty Cognition in Large Reasoning Models
  7. 2026Thinking Past the Answer: Evaluating Harmful Overthinking in Large Reasoning Models
  8. 2025What makes Reasoning Models Different? Follow the Reasoning Leader for Efficient Decoding