mlsanityData to 14 Oct 2026

What Characterizes Effective Reasoning? Revisiting Length, Review, and Structure of CoT

arXiv 2509.19284 ↗First posted 23 Sep 2025cs.LGcited by 25

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Large reasoning models (LRMs) spend substantial test-time compute on long chain-of-thought (CoT) traces, but what *characterizes* an effective CoT remains unclear. While prior work reports gains from lengthening CoTs and increasing review (revisiting earlier steps) via appended *wait* tokens, recent studies suggest that shorter thinking can outperform longer traces. We therefore conduct a systematic evaluation across ten LRMs on math and scientific reasoning. Contrary to the "longer-is-better" narrative, we find that both naive CoT lengthening and increased review are associated with *lower* accuracy. As CoT unfolds step by step, token-level metrics can conflate verbosity with process quality. We introduce a graph view of CoT to extract structure and identify a single statistic-the *Failed-Step Fraction (FSF)*, the fraction of steps in abandoned branches-that consistently outpredicts length and review ratio for correctness across models. To probe causality, we design two interventions. First, we rank candidate CoTs by each metric at test time, where FSF yields the largest pass@1 gains; second, we edit CoTs to remove failed branches, which significantly improves accuracy, indicating that failed branches bias subsequent reasoning. Taken together, these results characterize effective CoTs as those that *fail less* and support *structure-aware* test-time scaling over indiscriminately generating long CoT.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 31 of its 40 references that are in the map. First posted on arXiv 23 Sep 2025.

Builds on

31 of 40 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025What Makes a Good Reasoning Chain? Uncovering Structural Patterns in Long Chain-of-Thought Reasoning28 May
  2. 2025Fractured Chain-of-Thought Reasoning19 May
  3. 2025Reasoning Models Don't Always Say What They Think8 May
  4. 2025When More is Less: Understanding Chain-of-Thought Length in LLMs11 Feb
  5. 2023Measuring Faithfulness in Chain-of-Thought Reasoning17 Jul

Other references · 26

  1. 2025Qwen3 Technical Reportcited by 8,820
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  3. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  4. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
  5. 2025s1: Simple test-time scalingcited by 1,341
  6. 2024OpenAI o1 System Cardcited by 1,244
  7. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMscited by 570
  8. 2025Kimi K2: Open Agentic Intelligencecited by 567
  9. 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRscited by 377
  10. 2025Reasoning Models Know When They're Right: Probing Hidden States for Self-Verificationcited by 159
  11. 2024Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoningcited by 110
  12. 2025DeepSeek-R1 Thoughtology: Let's think about LLM Reasoningcited by 92
14 more
  1. 2025Don't Overthink it. Preferring Shorter Thinking Chains for Improved LLM Reasoningcited by 85
  2. 2025Wait, We Don't Need to "Wait"! Removing Thinking Tokens Improves Reasoning Efficiencycited by 66
  3. 2025Magistralcited by 60
  4. 2025Does Thinking More always Help? Mirage of Test-Time Scaling in Reasoning Modelscited by 53
  5. 2025Topology of Reasoning: Understanding Large Reasoning Models through Reasoning Graph Propertiescited by 29
  6. 2024HARP: A challenging human-annotated math reasoning benchmarkcited by 27
  7. 2025Beyond 'Aha!': Toward Systematic Meta-Abilities Alignment in Large Reasoning Modelscited by 18
  8. 2025Is That Your Final Answer? Test-Time Scaling Improves Selective Question Answeringcited by 17
  9. 2025RL of Thoughts: Navigating LLM Reasoning with Inference-time Reinforcement Learningcited by 16
  10. 2025Knowledge or Reasoning? A Close Look at How LLMs Think Across Domainscited by 14
  11. 2025Don't Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Modelscited by 11
  12. 2025How Far Are We from Optimal Reasoning Efficiency?cited by 8
  13. 2025It's Not That Simple. An Analysis of Simple Test-Time Scalingcited by 5
  14. 2025Learning a Continue-Thinking Token for Enhanced Test-Time Scalingcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 22.

25 papers.

Sort:

By area:

  1. 2026Reinforcing Chain-of-Thought Reasoning with Self-Evolving Rubricscited by 26
  2. 2026Thinking to Recall: How Reasoning Unlocks Parametric Knowledge in LLMscited by 16
  3. 2025Don't Waste Mistakes: Leveraging Negative RL-Groups via Confidence Reweightingcited by 12
  4. 2026The Hot Mess of AI: How Does Misalignment Scale With Model Intelligence and Task Complexity?cited by 10
  5. 2026Measuring Progress in Reasoning Toward Mathematical Discovery with Automatic Verificationcited by 9
  6. 2025Schoenfeld's Anatomy of Mathematical Reasoning by Language Modelscited by 8
  7. 2025ReJump: A Tree-Jump Representation for Analyzing and Improving LLM Reasoningcited by 5
  8. 2026Contextual Drag: How Errors in the Context Affect LLM Reasoningcited by 4
  9. 2026What Makes Good Multilingual Reasoning? Disentangling Traces with Measurable Featurescited by 4
  10. 2026Effective Reasoning Chains Reduce Intrinsic Dimensionalitycited by 3
  11. 2026Reasoning Shift: How Context Silently Shortens LLM Reasoningcited by 3
  12. 2026Reliable Chain-of-Thought via Prefix Consistencycited by 3
13 more
  1. 2026SARL: Label-Free Reinforcement Learning by Rewarding Reasoning Topologycited by 2
  2. 2026Cognitive Episodes in LLM Reasoning Traces Enable Interpretable Human Item Difficulty Predictioncited by 2
  3. 2026Reasoning about Reasoning: BAPO Bounds on Chain-of-Thought Token Complexity in LLMscited by 1
  4. 2026Boule or Baguette? A Study on Task Topology, Length Generalization, and the Benefit of Reasoning Tracescited by 1
  5. 2026VisRef: Visual Refocusing while Thinking Improves Test-Time Scaling in Multi-Modal Large Reasoning Modelscited by 1
  6. 2026Reasoning Models Know What's Important, and Encode It in Their Activationscited by 1
  7. 2026Think Longer to Explore Deeper: Learn to Explore In-Context via Length-Incentivized Reinforcement Learningcited by 0
  8. 2026Graph-Based Chain-of-Thought Pruning for Reducing Redundant Reflections in Reasoning LLMscited by 0
  9. 2026PathCal: State-Aware Reflection-Marker Calibration for Efficient Reasoningcited by 0
  10. 2026Reasoning Structure of Large Language Modelscited by 0
  11. 2026REChart: Reasoning-Efficient Chart Editing with Large Reasoning Modelscited by 0
  12. 2026Scaling of Capability and Efficiency at Inference Time in Large Reasoning Modelscited by 0
  13. 2026Facts Don't Speak Louder than Words: The Behavioral Essence of Long-CoT Distillationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026The Potential of CoT for Reasoning: A Closer Look at Trace Dynamics
  2. 2026CoTJudger: A Graph-Driven Framework for Automatic Evaluation of Chain-of-Thought Efficiency and Redundancy in LRMs
  3. 2025Reasoning with OmniThought: A Large CoT Dataset with Verbosity and Cognitive Difficulty Annotations
  4. 2025Performative Thinking? The Brittle Correlation Between CoT Length and Problem Complexity
  5. 2025Investigating CoT Monitorability in Large Reasoning Models
  6. 2024The Impact of Reasoning Step Length on Large Language Models
  7. 2025Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoning
  8. 2025Can Aha Moments Be Fake? Towards Quantifying Decorative and True Thinking in Chain-of-Thought