The First Impression Problem: Internal Bias Triggers Overthinking in Reasoning Models
On the map: LLM reasoning → Test-time compute and efficient thinking
Abstract (arXiv)
Reasoning models often exhibit overthinking, characterized by redundant reasoning steps. We identify internal bias elicited by the input question as a key trigger of such behavior. Upon encountering a problem, the model immediately forms a preliminary guess about the answer, which we term an internal bias since it may not be explicitly generated, and it arises without systematic reasoning. When this guess conflicts with its subsequent reasoning, the model tends to engage in excessive reflection, resulting in wasted computation. We validate the association between internal bias and overthinking across multiple models and diverse reasoning tasks. To demonstrate the causal relationship more rigorously, we conduct two counterfactual interventions, showing that removing the input question after the model reduces the redundant reasoning across various complex reasoning tasks, and manually injecting bias affects overthinking accordingly. Further interpretability experiments suggest that excessive attention to the input question serves as a key mechanism through which internal bias influences subsequent reasoning trajectories. Finally, we evaluated several methods aimed at mitigating overthinking, yet the influence of internal bias persisted under all conditions.
Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 28 of its 45 references that are in the map. First posted on arXiv 22 May 2025.
Builds on
28 of 45 references.
Closest ancestors · 10
Same direction, newest first
- 2025Learn to Reason Efficiently with Adaptive Length-based Reward Shaping21 May
- 2025Long-Short Chain-of-Thought Mixture Supervised Fine-Tuning Eliciting Efficient Reasoning in Large Language Models6 May
- 2025Between Underthinking and Overthinking: An Empirical Study of Reasoning Length and correctness in LLMs30 Apr
- 2025Dynamic Early Exit in Reasoning Models22 Apr
- 2025Reasoning Models Can Be Effective Without Thinking14 Apr
- 2025SEAL: Steerable Reasoning Calibration of Large Language Models for Free7 Apr
- 2025Reasoning Models Know When They're Right: Probing Hidden States for Self-Verification7 Apr
- 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models20 Mar
- 2025DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Models6 Mar
- 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs30 Dec
Other references · 18
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2024Qwen2.5 Technical Reportcited by 4,942
- 2024OpenAI o1 System Cardcited by 1,244
- 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
- 2024DeepSeek-V3 Technical Reportcited by 811
- 2025Reasoning Models Don't Always Say What They Thinkcited by 415
- 2024Alignment faking in large language modelscited by 360
- 2025Chain of Draft: Thinking Faster by Writing Lesscited by 297
- 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 293
- 2025Training Language Models to Reason Efficientlycited by 247
- 2024Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Modelscited by 178
- 2025Chain-of-Thought Reasoning In The Wild Is Not Always Faithfulcited by 170
6 more
- 2022Learning to Break the Loop: Analyzing and Mitigating Repetitions for Neural Text Generationcited by 94
- 2023System 2 Attention (is something you might need too)cited by 89
- 2025Ada-R1: Hybrid-CoT via Bi-Level Adaptive Reasoning Optimizationcited by 33
- 2024Controllable Context Sensitivity and the Knob Behind Itcited by 23
- 2025Identifying and Mitigating the Influence of the Prior Distribution in Large Language Modelscited by 4
- 2025SCoRE: Benchmarking Long-Chain Reasoning in Commonsense Scenarioscited by 3
Built on by
Citing papers per year: 2025: 5; 2026: 5.
10 papers.
- 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 25
- 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Surveycited by 18
- 2026Towards a Mechanistic Understanding of Large Reasoning Models: A Survey of Training, Inference, and Failurescited by 6
- 2025Med-REFL: Medical Reasoning Enhancement via Self-Corrected Fine-grained Reflectioncited by 3
- 2025Answering the Unanswerable Is to Err Knowingly: Analyzing and Mitigating Abstention Failures in Large Reasoning Modelscited by 3
- 2025CoreThink: A Symbolic Reasoning Layer to reason over Long Horizon Tasks with LLMscited by 3
- 2026TREK: A Travel Reasoning and Evaluation Kit for LLM Agents in Complex Trip Planningcited by 2
- 2026Implicit Compression Regularization: Concise Reasoning via Internal Shorter Distributions in RL Post-Trainingcited by 1
- 2026Graph-Based Chain-of-Thought Pruning for Reducing Redundant Reflections in Reasoning LLMscited by 0
- 2026Scaling of Capability and Efficiency at Inference Time in Large Reasoning Modelscited by 0
- 2026Scaling of Capability and Efficiency at Inference Time in Large Reasoning Models22 Sep 2026
- 2026TREK: A Travel Reasoning and Evaluation Kit for LLM Agents in Complex Trip Planning29 Jul 2026
- 2026Implicit Compression Regularization: Concise Reasoning via Internal Shorter Distributions in RL Post-Training8 May 2026
- 2026Graph-Based Chain-of-Thought Pruning for Reducing Redundant Reflections in Reasoning LLMs7 Apr 2026
- 2026Towards a Mechanistic Understanding of Large Reasoning Models: A Survey of Training, Inference, and Failures11 Jan 2026
- 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models4 Sep 2025
- 2025CoreThink: A Symbolic Reasoning Layer to reason over Long Horizon Tasks with LLMs31 Aug 2025
- 2025Answering the Unanswerable Is to Err Knowingly: Analyzing and Mitigating Abstention Failures in Large Reasoning Models26 Aug 2025
- 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey13 Jul 2025
- 2025Med-REFL: Medical Reasoning Enhancement via Self-Corrected Fine-grained Reflection11 Jun 2025
Similar, not cited
Close in wording, no citation either way
- 2025Revisiting Overthinking in Long Chain-of-Thought from the Perspective of Self-Doubt
- 2026Thinking Past the Answer: Evaluating Harmful Overthinking in Large Reasoning Models
- 2026The Long Road to the Same Answer: Cognitive Bias Under Escalating Reasoning Budgets in Large Language Models
- 2025When Can Large Reasoning Models Save Thinking? Mechanistic Analysis of Behavioral Divergence in Reasoning
- 2025The Danger of Overthinking: Examining the Reasoning-Action Dilemma in Agentic Tasks
- 2026BiasTrace: Linking Reasoning Behaviours to Biased Outputs in LLMs
- 2025Missing Premise exacerbates Overthinking: Are Reasoning Models losing Critical Thinking Skill?
- 2026Efficient Reasoning with Balanced Thinking