# Don't Overthink it. Preferring Shorter Thinking Chains for Improved LLM Reasoning

arXiv: https://arxiv.org/abs/2505.17813 · HTML: https://mlsanity.com/paper/2505.17813/

## Key facts

- Cited by 85 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 25 of its 33 references that are in the map.
- First posted on arXiv 23 May 2025.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 25 of 25)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [s1: Simple test-time scaling](https://mlsanity.com/paper/2501.19393/) (2025)
- [LIMO: Less is More for Reasoning](https://mlsanity.com/paper/2502.03387/) (2025)
- [Training Language Models to Reason Efficiently](https://mlsanity.com/paper/2502.04463/) (2025)
- [Dynamic Early Exit in Reasoning Models](https://mlsanity.com/paper/2504.15895/) (2025)
- [When More is Less: Understanding Chain-of-Thought Length in LLMs](https://mlsanity.com/paper/2502.07266/) (2025)
- [SPoC: Search-based Pseudocode to Code](https://mlsanity.com/paper/1906.04908/) (2019)
- [Phi-4-reasoning Technical Report](https://mlsanity.com/paper/2504.21318/) (2025)
- [C3oT: Generating Shorter Chain-of-Thought without Compromising Effectiveness](https://mlsanity.com/paper/2412.11664/) (2024)
- [A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systems](https://mlsanity.com/paper/2504.09037/) (2025)
- [Concise Thoughts: Impact of Output Length on LLM Reasoning and Cost](https://mlsanity.com/paper/2407.19825/) (2024)
- [Concise Reasoning via Reinforcement Learning](https://mlsanity.com/paper/2504.05185/) (2025)
- [The Larger the Better? Improved LLM Code-Generation via Budget Reallocation](https://mlsanity.com/paper/2404.00725/) (2024)
- [Z1: Efficient Test-time Scaling with Code](https://mlsanity.com/paper/2504.00810/) (2025)
- [THOUGHTTERMINATOR: Benchmarking, Calibrating, and Mitigating Overthinking in Reasoning Models](https://mlsanity.com/paper/2504.13367/) (2025)
- [Retro-Search: Exploring Untaken Paths for Deeper and Efficient Reasoning](https://mlsanity.com/paper/2504.04383/) (2025)
- [Optimizing Anytime Reasoning via Budget Relative Policy Optimization](https://mlsanity.com/paper/2505.13438/) (2025)
- [Rethinking Inference-Time Scaling: Efficiency Limits and Linguistic Signals](https://mlsanity.com/paper/2504.14047/) (2025)

## Built on by (most cited first, 30 of 85)

- [Deep Think with Confidence](https://mlsanity.com/paper/2508.15260/) (2025)
- [CWM: An Open-Weights LLM for Research on Code Generation with World Models](https://mlsanity.com/paper/2510.02387/) (2025)
- [Sample More to Think Less: Group Filtered Policy Optimization for Concise Reasoning](https://mlsanity.com/paper/2508.09726/) (2025)
- [Inverse Scaling in Test-Time Compute](https://mlsanity.com/paper/2507.14417/) (2025)
- [Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Models](https://mlsanity.com/paper/2508.02120/) (2025)
- [Jigsaw-R1: A Study of Rule-based Visual Reinforcement Learning with Jigsaw Puzzles](https://mlsanity.com/paper/2505.23590/) (2025)
- [What Characterizes Effective Reasoning? Revisiting Length, Review, and Structure of CoT](https://mlsanity.com/paper/2509.19284/) (2025)
- [Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens](https://mlsanity.com/paper/2602.13517/) (2026)
- [Trace Length is a Simple Uncertainty Signal in Reasoning Models](https://mlsanity.com/paper/2510.10409/) (2025)
- [Does Your Reasoning Model Implicitly Know When to Stop Thinking?](https://mlsanity.com/paper/2602.08354/) (2026)
- [Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey](https://mlsanity.com/paper/2507.09662/) (2025)
- [Thinking to Recall: How Reasoning Unlocks Parametric Knowledge in LLMs](https://mlsanity.com/paper/2603.09906/) (2026)
- [Slim-SC: Thought Pruning for Efficient Scaling with Self-Consistency](https://mlsanity.com/paper/2509.13990/) (2025)
- [Leash: Adaptive Length Penalty and Reward Shaping for Efficient Large Reasoning Model](https://mlsanity.com/paper/2512.21540/) (2025)
- [The Art of Scaling Test-Time Compute for Large Language Models](https://mlsanity.com/paper/2512.02008/) (2025)
- [PiCSAR: Probabilistic Confidence Selection And Ranking for Reasoning Chains](https://mlsanity.com/paper/2508.21787/) (2025)
- [Do LLMs Really Need 10+ Thoughts for "Find the Time 1000 Days Later"? Towards Structural Understanding of LLM Overthinking](https://mlsanity.com/paper/2510.07880/) (2025)
- [The Hot Mess of AI: How Does Misalignment Scale With Model Intelligence and Task Complexity?](https://mlsanity.com/paper/2601.23045/) (2026)
- [Quantized Reasoning Models Think They Need to Think Longer, but They Do Not](https://mlsanity.com/paper/2606.00206/) (2026)
- [Performative Thinking? The Brittle Correlation Between CoT Length and Problem Complexity](https://mlsanity.com/paper/2509.07339/) (2025)
- [Tracing the Traces: Latent Temporal Signals for Efficient and Accurate Reasoning](https://mlsanity.com/paper/2510.10494/) (2025)
- [LaSeR: Reinforcement Learning with Last-Token Self-Rewarding](https://mlsanity.com/paper/2510.14943/) (2025)
- [Reasoning Under Pressure: How do Training Incentives Influence Chain-of-Thought Monitorability?](https://mlsanity.com/paper/2512.00218/) (2025)
- [Latent Reasoning with Supervised Thinking States](https://mlsanity.com/paper/2602.08332/) (2026)
- [Reducing Latency of LLM Search Agent via Speculation-based Algorithm-System Co-Design](https://mlsanity.com/paper/2511.20048/) (2025)
- [Neural Chain-of-Thought Search: Searching the Optimal Reasoning Path to Enhance Large Language Models](https://mlsanity.com/paper/2601.11340/) (2026)
- [OPD-V: Visual On-Policy Self-Distillation with Modality Balance](https://mlsanity.com/paper/2608.05131/) (2026)
- [Reward Modeling for Reinforcement Learning-Based LLM Reasoning: Design, Challenges, and Evaluation](https://mlsanity.com/paper/2602.09305/) (2026)
- [Procedural Knowledge at Scale Improves Reasoning](https://mlsanity.com/paper/2604.01348/) (2026)
- [Building Coding Agents via Entropy-Enhanced Multi-Turn Preference Optimization](https://mlsanity.com/paper/2509.12434/) (2025)

## Abstract (arXiv)

Reasoning large language models (LLMs) heavily rely on scaling test-time compute to perform complex reasoning tasks by generating extensive "thinking" chains. While demonstrating impressive results, this approach incurs significant computational costs and inference time. In this work, we challenge the assumption that long thinking chains results in better reasoning capabilities. We first demonstrate that shorter reasoning chains within individual questions are significantly more likely to yield correct answers - up to 34.5% more accurate than the longest chain sampled for the same question. Based on these results, we suggest short-m@k, a novel reasoning LLM inference method. Our method executes k independent generations in parallel and halts computation once the first m thinking processes are done. The final answer is chosen using majority voting among these m chains. Basic short-1@k demonstrates similar or even superior performance over standard majority voting in low-compute settings - using up to 40% fewer thinking tokens. short-3@k, while slightly less efficient than short-1@k, consistently surpasses majority voting across all compute budgets, while still being substantially faster (up to 33% wall time reduction). To further validate our findings, we finetune LLMs using short, long, and randomly selected reasoning chains. We then observe that training on the shorter ones leads to better performance. Our findings suggest rethinking current methods of test-time compute in reasoning LLMs, emphasizing that longer "thinking" does not necessarily translate to improved performance and can, counter-intuitively, lead to degraded results.