mlsanityData to 7 Oct 2026

Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens

arXiv 2602.13517 ↗First posted 13 Feb 2026cs.CLcited by 21

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Large language models (LLMs) have demonstrated impressive reasoning capabilities by scaling test-time compute via long Chain-of-Thought (CoT). However, recent findings suggest that raw token counts are unreliable proxies for reasoning quality: increased generation length does not consistently correlate with accuracy and may instead signal "overthinking," leading to performance degradation. In this work, we quantify inference-time effort by identifying deep-thinking tokens -- tokens where internal predictions undergo significant revisions in deeper model layers prior to convergence. Across four challenging mathematical and scientific benchmarks (AIME 24/25, HMMT 25, and GPQA-diamond) and a diverse set of reasoning-focused models (GPT-OSS, DeepSeek-R1, and Qwen3), we show that deep-thinking ratio (the proportion of deep-thinking tokens in a generated sequence) exhibits a robust and consistently positive correlation with accuracy, substantially outperforming both length-based and confidence-based baselines. Leveraging this insight, we introduce Think@n, a test-time scaling strategy that prioritizes samples with high deep-thinking ratios. We demonstrate that Think@n matches or exceeds standard self-consistency performance while significantly reducing inference costs by enabling the early rejection of unpromising generations based on short prefixes.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 29 of its 53 references that are in the map. First posted on arXiv 13 Feb 2026.

Builds on

29 of 53 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025Deep Think with Confidence21 Aug
  2. 2025gpt-oss-120b & gpt-oss-20b Model Card8 Aug
  3. 2025Inverse Scaling in Test-Time Compute19 Jul
  4. 2025Test-Time Scaling with Repeated Sampling Improves Multilingual Text Generation28 May
  5. 2025First Finish Search: Efficient Test-Time Scaling in Large Language Models23 May
  6. 2025Don't Overthink it. Preferring Shorter Thinking Chains for Improved LLM Reasoning23 May
  7. 2025Between Underthinking and Overthinking: An Empirical Study of Reasoning Length and correctness in LLMs30 Apr
  8. 2025Inference-Time Scaling for Complex Tasks: Where We Stand and What Lies Ahead31 Mar
  9. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models20 Mar
  10. 2025Scalable Best-of-N Selection for Large Language Models via Self-Certainty25 Feb
  11. 2025Demystifying Long Chain-of-Thought Reasoning in LLMs5 Feb

Other references · 18

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  4. 2024OpenAI o1 System Cardcited by 1,220
  5. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  6. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
  7. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
  8. 2025MathArena: Evaluating LLMs on Uncontaminated Math Competitionscited by 275
  9. 2019Depth-Adaptive Transformercited by 227
  10. 2024Evaluation of OpenAI o1: Opportunities and Challenges of AGIcited by 127
  11. 2025Sample More to Think Less: Group Filtered Policy Optimization for Concise Reasoningcited by 75
  12. 2024Archon: An Architecture Search Framework for Inference-Time Techniquescited by 59
6 more
  1. 2024The Limits of Inference Scaling Through Resamplingcited by 51
  2. 2025Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodalitycited by 49
  3. 2025Do Language Models Use Their Depth Efficiently?cited by 48
  4. 2025Detection and Mitigation of Hallucination in Large Reasoning Models: A Mechanistic Perspectivecited by 32
  5. 2025Shrinking the Generation-Verification Gap with Weak Verifierscited by 21
  6. 2020BERT's output layer recognizes all hidden layers? Some Intriguing Phenomena and a simple way to boost BERTcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 21.

21 papers.

Sort:

By area:

  1. 2026Co-Evolving Skill Generation and Policy Optimizationcited by 5
  2. 2026On the Role of Reasoning Patterns in the Generalization Discrepancy of Long Chain-of-Thought Supervised Fine-Tuningcited by 4
  3. 2026Latent State Design for World Models under Sufficiency Constraintscited by 3
  4. 2026When Do LLMs Reason? A Dynamical Systems View via Entropy Phase Transitionscited by 2
  5. 2026Reasoning Models Know What's Important, and Encode It in Their Activationscited by 1
  6. 2026GRASP: Learning to Ground Social Reasoning in Multi-Person Non-Verbal Interactionscited by 1
  7. 2026Human vs Machine Mathematical Difficulty on Project Euler: An Experimental Analysiscited by 1
  8. 2026Spatiotemporal Hidden-State Dynamics as a Signature of Internal Reasoning in Large Language Modelscited by 0
  9. 2026Stateful Reasoning via Insight Replaycited by 0
  10. 2026Effort as Ceiling, Not Dial: Reasoning Budget Does Not Modulate Cognitive Cost Alignment Between Humans and Large Reasoning Modelscited by 0
  11. 2026Integrated and Cross-Architecture Interpretation of LLM Reasoningcited by 0
  12. 2026QLPO: Quadrant-weighted Sampling for Length-aware Policy Optimizationcited by 0
9 more
  1. 2026ThinkReset: Learnable Intermediate Interface Construction for Bounded-Context Long-Horizon Reasoningcited by 0
  2. 2026How Hard Does It Think? Analyzing Step-Aware Reasoning Energy in LLM Chain-of-Thought Trajectoriescited by 0
  3. 2026On the use of foundation models in cognitive sciencecited by 0
  4. 2026Expert-validated STEM QAcited by 0
  5. 2026Test-Time Scaling for Video Diffusion Models via Diagnosis-Guided Candidate Recyclingcited by 0
  6. 2026Thinking effort aligns between humans and reasoning models in abductive reasoningcited by 0
  7. 2026Improving Test-Time Scaling with Adaptive Looped Transformerscited by 0
  8. 2026S3: Spectral Null-Space Swap Makes Reasoning Models Efficientcited by 0
  9. 2026HeadEdit: Calibrating Language Model Behavior Through the Frozen Unembedding Matrixcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoning
  2. 2025Think Twice: Enhancing LLM Reasoning by Scaling Multi-round Test-time Thinking
  3. 2026ThinkBooster: A Unified Framework for Seamless Test-Time Scaling of LLM Reasoning
  4. 2025The Art of Scaling Test-Time Compute for Large Language Models
  5. 2026ReEfBench: Quantifying the Reasoning Efficiency of LLMs
  6. 2024Extending Token Computation for LLM Reasoning
  7. 2025Beyond Chains of Thought: Benchmarking Latent-Space Reasoning Abilities in Large Language Models
  8. 2025Prompting Test-Time Scaling Is A Strong LLM Reasoning Data Augmentation