# What Characterizes Effective Reasoning? Revisiting Length, Review, and Structure of CoT

arXiv: https://arxiv.org/abs/2509.19284 · HTML: https://mlsanity.com/paper/2509.19284/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- Builds on 31 of its 40 references that are in the map.
- First posted on arXiv 23 Sep 2025.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)

## Builds on (most cited first, 30 of 31)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [gpt-oss-120b & gpt-oss-20b Model Card](https://mlsanity.com/paper/2508.10925/) (2025)
- [s1: Simple test-time scaling](https://mlsanity.com/paper/2501.19393/) (2025)
- [OpenAI o1 System Card](https://mlsanity.com/paper/2412.16720/) (2024)
- [Measuring Faithfulness in Chain-of-Thought Reasoning](https://mlsanity.com/paper/2307.13702/) (2023)
- [Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs](https://mlsanity.com/paper/2412.21187/) (2024)
- [Kimi K2: Open Agentic Intelligence](https://mlsanity.com/paper/2507.20534/) (2025)
- [Reasoning Models Don't Always Say What They Think](https://mlsanity.com/paper/2505.05410/) (2025)
- [Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs](https://mlsanity.com/paper/2503.01307/) (2025)
- [When More is Less: Understanding Chain-of-Thought Length in LLMs](https://mlsanity.com/paper/2502.07266/) (2025)
- [Reasoning Models Know When They're Right: Probing Hidden States for Self-Verification](https://mlsanity.com/paper/2504.05419/) (2025)
- [Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning](https://mlsanity.com/paper/2412.09078/) (2024)
- [DeepSeek-R1 Thoughtology: Let's think about LLM Reasoning](https://mlsanity.com/paper/2504.07128/) (2025)
- [Don't Overthink it. Preferring Shorter Thinking Chains for Improved LLM Reasoning](https://mlsanity.com/paper/2505.17813/) (2025)
- [Wait, We Don't Need to "Wait"! Removing Thinking Tokens Improves Reasoning Efficiency](https://mlsanity.com/paper/2506.08343/) (2025)
- [Magistral](https://mlsanity.com/paper/2506.10910/) (2025)
- [Does Thinking More always Help? Mirage of Test-Time Scaling in Reasoning Models](https://mlsanity.com/paper/2506.04210/) (2025)
- [Topology of Reasoning: Understanding Large Reasoning Models through Reasoning Graph Properties](https://mlsanity.com/paper/2506.05744/) (2025)
- [HARP: A challenging human-annotated math reasoning benchmark](https://mlsanity.com/paper/2412.08819/) (2024)
- [What Makes a Good Reasoning Chain? Uncovering Structural Patterns in Long Chain-of-Thought Reasoning](https://mlsanity.com/paper/2505.22148/) (2025)
- [Beyond 'Aha!': Toward Systematic Meta-Abilities Alignment in Large Reasoning Models](https://mlsanity.com/paper/2505.10554/) (2025)
- [Is That Your Final Answer? Test-Time Scaling Improves Selective Question Answering](https://mlsanity.com/paper/2502.13962/) (2025)
- [RL of Thoughts: Navigating LLM Reasoning with Inference-time Reinforcement Learning](https://mlsanity.com/paper/2505.14140/) (2025)
- [Knowledge or Reasoning? A Close Look at How LLMs Think Across Domains](https://mlsanity.com/paper/2506.02126/) (2025)
- [Don't Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Models](https://mlsanity.com/paper/2505.21765/) (2025)
- [How Far Are We from Optimal Reasoning Efficiency?](https://mlsanity.com/paper/2506.07104/) (2025)
- [Fractured Chain-of-Thought Reasoning](https://mlsanity.com/paper/2505.12992/) (2025)
- [It's Not That Simple. An Analysis of Simple Test-Time Scaling](https://mlsanity.com/paper/2507.14419/) (2025)

## Built on by (most cited first, 25 of 25)

- [Reinforcing Chain-of-Thought Reasoning with Self-Evolving Rubrics](https://mlsanity.com/paper/2602.10885/) (2026)
- [Thinking to Recall: How Reasoning Unlocks Parametric Knowledge in LLMs](https://mlsanity.com/paper/2603.09906/) (2026)
- [Don't Waste Mistakes: Leveraging Negative RL-Groups via Confidence Reweighting](https://mlsanity.com/paper/2510.08696/) (2025)
- [The Hot Mess of AI: How Does Misalignment Scale With Model Intelligence and Task Complexity?](https://mlsanity.com/paper/2601.23045/) (2026)
- [Measuring Progress in Reasoning Toward Mathematical Discovery with Automatic Verification](https://mlsanity.com/paper/2603.15617/) (2026)
- [Schoenfeld's Anatomy of Mathematical Reasoning by Language Models](https://mlsanity.com/paper/2512.19995/) (2025)
- [ReJump: A Tree-Jump Representation for Analyzing and Improving LLM Reasoning](https://mlsanity.com/paper/2512.00831/) (2025)
- [Contextual Drag: How Errors in the Context Affect LLM Reasoning](https://mlsanity.com/paper/2602.04288/) (2026)
- [What Makes Good Multilingual Reasoning? Disentangling Traces with Measurable Features](https://mlsanity.com/paper/2604.04720/) (2026)
- [Effective Reasoning Chains Reduce Intrinsic Dimensionality](https://mlsanity.com/paper/2602.09276/) (2026)
- [Reasoning Shift: How Context Silently Shortens LLM Reasoning](https://mlsanity.com/paper/2604.01161/) (2026)
- [Reliable Chain-of-Thought via Prefix Consistency](https://mlsanity.com/paper/2605.07654/) (2026)
- [SARL: Label-Free Reinforcement Learning by Rewarding Reasoning Topology](https://mlsanity.com/paper/2603.27977/) (2026)
- [Cognitive Episodes in LLM Reasoning Traces Enable Interpretable Human Item Difficulty Prediction](https://mlsanity.com/paper/2606.28186/) (2026)
- [Reasoning about Reasoning: BAPO Bounds on Chain-of-Thought Token Complexity in LLMs](https://mlsanity.com/paper/2602.02909/) (2026)
- [Boule or Baguette? A Study on Task Topology, Length Generalization, and the Benefit of Reasoning Traces](https://mlsanity.com/paper/2602.14404/) (2026)
- [VisRef: Visual Refocusing while Thinking Improves Test-Time Scaling in Multi-Modal Large Reasoning Models](https://mlsanity.com/paper/2603.00207/) (2026)
- [Reasoning Models Know What's Important, and Encode It in Their Activations](https://mlsanity.com/paper/2604.18307/) (2026)
- [Think Longer to Explore Deeper: Learn to Explore In-Context via Length-Incentivized Reinforcement Learning](https://mlsanity.com/paper/2602.11748/) (2026)
- [Graph-Based Chain-of-Thought Pruning for Reducing Redundant Reflections in Reasoning LLMs](https://mlsanity.com/paper/2604.05643/) (2026)
- [PathCal: State-Aware Reflection-Marker Calibration for Efficient Reasoning](https://mlsanity.com/paper/2605.23074/) (2026)
- [Reasoning Structure of Large Language Models](https://mlsanity.com/paper/2606.03883/) (2026)
- [REChart: Reasoning-Efficient Chart Editing with Large Reasoning Models](https://mlsanity.com/paper/2608.17414/) (2026)
- [Scaling of Capability and Efficiency at Inference Time in Large Reasoning Models](https://mlsanity.com/paper/2609.27166/) (2026)
- [Facts Don't Speak Louder than Words: The Behavioral Essence of Long-CoT Distillation](https://mlsanity.com/paper/2610.17607/) (2026)

## Abstract (arXiv)

Large reasoning models (LRMs) spend substantial test-time compute on long chain-of-thought (CoT) traces, but what *characterizes* an effective CoT remains unclear. While prior work reports gains from lengthening CoTs and increasing review (revisiting earlier steps) via appended *wait* tokens, recent studies suggest that shorter thinking can outperform longer traces. We therefore conduct a systematic evaluation across ten LRMs on math and scientific reasoning. Contrary to the "longer-is-better" narrative, we find that both naive CoT lengthening and increased review are associated with *lower* accuracy. As CoT unfolds step by step, token-level metrics can conflate verbosity with process quality. We introduce a graph view of CoT to extract structure and identify a single statistic-the *Failed-Step Fraction (FSF)*, the fraction of steps in abandoned branches-that consistently outpredicts length and review ratio for correctness across models. To probe causality, we design two interventions. First, we rank candidate CoTs by each metric at test time, where FSF yields the largest pass@1 gains; second, we edit CoTs to remove failed branches, which significantly improves accuracy, indicating that failed branches bias subsequent reasoning. Taken together, these results characterize effective CoTs as those that *fail less* and support *structure-aware* test-time scaling over indiscriminately generating long CoT.