mlsanityData to 7 Oct 2026

Thought calibration: Efficient and confident test-time scaling

arXiv 2505.18404 ↗First posted 23 May 2025cs.LG cs.AIcited by 20

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Reasoning large language models achieve impressive test-time scaling by thinking for longer, but this performance gain comes at significant compute cost. Directly limiting test-time budget hurts overall performance, but not all problems are equally difficult. We propose thought calibration to decide dynamically when thinking can be terminated. To calibrate our decision rule, we view a language model's growing body of thoughts as a nested sequence of reasoning trees, where the goal is to identify the point at which novel reasoning plateaus. We realize this framework through lightweight probes that operate on top of the language model's hidden representations, which are informative of both the reasoning structure and overall consistency of response. Based on three reasoning language models and four datasets, thought calibration preserves model performance with up to a 60% reduction in thinking tokens on in-distribution data, and up to 20% in out-of-distribution data.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 20 of its 51 references that are in the map. First posted on arXiv 23 May 2025.

Builds on

20 of 51 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Dynamic Early Exit in Reasoning Models22 Apr
  2. 2025Reasoning Models Can Be Effective Without Thinking14 Apr
  3. 2025Reasoning Models Know When They're Right: Probing Hidden States for Self-Verification7 Apr
  4. 2025ThinkPrune: Pruning Long Chain-of-Thought of LLMs via Reinforcement Learning2 Apr
  5. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models20 Mar
  6. 2025Efficient Test-Time Scaling via Self-Calibration25 Feb
  7. 2025s1: Simple test-time scaling31 Jan
  8. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs30 Dec
  9. 2024Token-Budget-Aware LLM Reasoning24 Dec

Other references · 11

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2020Scaling Laws for Neural Language Modelscited by 6,411
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  5. 2024Qwen2.5 Technical Reportcited by 4,713
  6. 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 777
  7. 2021Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Controlcited by 256
  8. 2025Training Language Models to Reason Efficientlycited by 242
  9. 2024Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMa-3 8Bcited by 156
  10. 2024Calibrating Reasoning in Language Models with Internal Consistencycited by 27
  11. 2025Temporal Consistency for LLM Reasoning Process Error Identificationcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 17.

20 papers.

Sort:

By area:

  1. 2026Dynamic Model Routing and Cascading for Efficient LLM Inference: A Surveycited by 16
  2. 2025E-valuator: Reliable Agent Verifiers with Sequential Hypothesis Testingcited by 14
  3. 2026Conformal Thinking: Risk Control for Reasoning on a Compute Budgetcited by 9
  4. 2026Entropy trajectory shape predicts LLM reasoning reliability: A diagnostic study of uncertainty dynamics in chain-of-thoughtcited by 6
  5. 2025ReJump: A Tree-Jump Representation for Analyzing and Improving LLM Reasoningcited by 5
  6. 2026Statistical Early Stopping for Reasoning Modelscited by 4
  7. 2026Online Reasoning Calibration: Test-Time Training Enables Generalizable Conformal LLM Reasoningcited by 4
  8. 2026From Reasoning Traces to Reusable Modules: Understanding Compositional Generalization in Language Model Reasoningcited by 3
  9. 2025Reliable LLM-Based Edge-Cloud-Expert Cascades for Telecom Knowledge Systemscited by 2
  10. 2026SHERLOC: Structured Diagnostic Localization for Code Repair Agentscited by 2
  11. 2026Prefix Sliding for efficient test-time scalingcited by 2
  12. 2026Process Supervision of Confidence Margin for Calibrated LLM Reasoningcited by 1
8 more
  1. 2026Conformal Language Modeling via Posterior Samplingcited by 1
  2. 2026When Does Learning to Stop Help? A Cost-Aware Study of Early Exits in Reasoning Modelscited by 1
  3. 2026Think Short, Defer Smart, Act, and Repeat: Calibrated Reasoning and Uncertainty-Aware Deferral for Edge LLM Agentscited by 1
  4. 2026Algebraic Quantum Intelligence: A New Framework for Reproducible Machine Creativitycited by 0
  5. 2026Primal Generation, Dual Judgment: Self-Training from Test-Time Scalingcited by 0
  6. 2026Online Safety Monitoring for LLMscited by 0
  7. 2026Code-MUE: Measuring Code LLMs' Uncertainty through Execution-based Semantic Interaction Graphscited by 0
  8. 2026Funnel of Thoughts: Efficient Test-Time Scaling via Early Voting and Rollout Pruningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025The Art of Scaling Test-Time Compute for Large Language Models
  2. 2026ThinkRetrieve: Retrieval-Augmented Reasoning Traces for Test-Time Scaling
  3. 2025The Evolution of Thought: Tracking LLM Overthinking via Reasoning Dynamics Analysis
  4. 2026When More Thinking Hurts: Overthinking in LLM Test-Time Compute Scaling
  5. 2025Understanding and Steering the Cognitive Behaviors of Reasoning Models at Test-Time
  6. 2026Efficient Test-Time Scaling via Temporal Reasoning Aggregation
  7. 2025Reasoning Models Better Express Their Confidence
  8. 2026Improving reasoning at inference time via uncertainty minimisation