mlsanityData to 7 Oct 2026

THINK-Bench: Evaluating Thinking Efficiency and Chain-of-Thought Quality of Large Reasoning Models

arXiv 2505.22113 ↗First posted 28 May 2025cs.CLcited by 18

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Large reasoning models (LRMs) have achieved impressive performance in complex tasks, often outperforming conventional large language models (LLMs). However, the prevalent issue of overthinking severely limits their computational efficiency. Overthinking occurs when models generate excessive and redundant tokens that contribute little to accurate outcomes, especially in simple tasks, resulting in a significant waste of computational resources. To systematically investigate this issue, we introduce Think-Bench, a benchmark designed to evaluate the reasoning efficiency of LRMs. We also propose novel efficiency metrics and conduct a comprehensive evaluation of various LRMs across multiple dimensions, including the reasoning process, outcome quality, and chain-of-thought (CoT) characteristics. Our analysis reveals that most LRMs exhibit overthinking in handling easy questions, generating unnecessarily lengthy reasoning chains. While many LRMs demonstrate high CoT quality, several suffer from low efficiency. We hope that Think-Bench can serve as a robust foundation for advancing research into LRMs.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 28 of its 42 references that are in the map. First posted on arXiv 28 May 2025.

Builds on

28 of 42 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025Dynamic Early Exit in Reasoning Models22 Apr
  2. 2025THOUGHTTERMINATOR: Benchmarking, Calibrating, and Mitigating Overthinking in Reasoning Models17 Apr
  3. 2025Missing Premise exacerbates Overthinking: Are Reasoning Models losing Critical Thinking Skill?9 Apr
  4. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models20 Mar
  5. 2025The Danger of Overthinking: Examining the Reasoning-Action Dilemma in Agentic Tasks12 Feb
  6. 2025s1: Simple test-time scaling31 Jan
  7. 2025Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs30 Jan
  8. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs30 Dec

Other references · 20

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  5. 2024Qwen2.5 Technical Reportcited by 4,713
  6. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  7. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  8. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 701
  9. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 684
  10. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  11. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 291
  12. 2024ProcessBench: Identifying Process Errors in Mathematical Reasoningcited by 277
8 more
  1. 2024Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoningcited by 136
  2. 2024Evaluation of OpenAI o1: Opportunities and Challenges of AGIcited by 127
  3. 2025MME-CoT: Benchmarking Chain-of-Thought in Large Multimodal Models for Reasoning Quality, Robustness, and Efficiencycited by 123
  4. 2023ARB: Advanced Reasoning Benchmark for Large Language Modelscited by 60
  5. 2025UGPhysics: A Comprehensive Benchmark for Undergraduate Physics Reasoning with Large Language Modelscited by 45
  6. 2024Multi-LogiEval: Towards Evaluating Multi-Step Logical Reasoning Ability of Large Language Modelscited by 38
  7. 2024SciKnowEval: Evaluating Multi-level Scientific Knowledge of Large Language Modelscited by 19
  8. 2024MiCEval: Unveiling Multimodal Chain of Thought's Quality via Image Description and Reasoning Stepscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 12.

18 papers.

Sort:

By area:

  1. 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Surveycited by 18
  2. 2025What Defines Good Reasoning in LLMs? Dissecting Reasoning Steps with Multi-Aspect Evaluationcited by 10
  3. 2025OckBench: Measuring the Efficiency of LLM Reasoningcited by 10
  4. 2026Neural Chain-of-Thought Search: Searching the Optimal Reasoning Path to Enhance Large Language Modelscited by 7
  5. 2025Do LLMs Overthink Basic Math Reasoning? Benchmarking the Accuracy-Efficiency Tradeoff in Language Modelscited by 5
  6. 2026What Makes Good Multilingual Reasoning? Disentangling Traces with Measurable Featurescited by 4
  7. 2026Exposing Weaknesses of Large Reasoning Models through Graph Algorithm Problemscited by 3
  8. 2025Strategic Reflectivism In Intelligent Systemscited by 0
  9. 2025AgroCoT: A Chain-of-Thought Benchmark for Evaluating Reasoning in Vision-Language Models for Agriculturecited by 0
  10. 2026EvalQReason: A Framework for Step-Level Reasoning Evaluation in Large Language Modelscited by 0
  11. 2026Structural Anchors and Reasoning Fragility:Understanding CoT Robustness in LLM4Codecited by 0
  12. 2026Distilling Long-CoT Reasoning through Collaborative Step-wise Multi-Teacher Decodingcited by 0
6 more
  1. 2026CRANE: Constrained Reasoning Injection for Code Agents via Nullspace Editingcited by 0
  2. 2026RecurGuard: Runtime Monitoring for Reasoning-Token Consumption Attackscited by 0
  3. 2026ThinkProbe: Beyond Accuracy -- Structural Profiling of Open-Ended LLM Reasoning Traces via Non-Generative Thought Graphscited by 0
  4. 2026TsuGO: Probing Search Efficiency in LLM Reasoning via Go Life-and-Death Problemscited by 0
  5. 2026Who Drives the Probability Game of VLMs? A Temporal Causal Drive Evaluation Frameworkcited by 0
  6. 2026When2Think: Learning Difficulty-Aware Length Control for Efficient Hybrid Reasoning Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Think or Not? Exploring Thinking Efficiency in Large Reasoning Models via an Information-Theoretic Lens
  2. 2025Efficient Inference for Large Reasoning Models: A Survey
  3. 2025Exploring the System 1 Thinking Capability of Large Reasoning Models
  4. 2026CoTJudger: A Graph-Driven Framework for Automatic Evaluation of Chain-of-Thought Efficiency and Redundancy in LRMs
  5. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyond
  6. 2025EffiReason-Bench: A Unified Benchmark for Evaluating and Advancing Efficient Reasoning in Large Language Models
  7. 2025The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
  8. 2023Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performance