# Thought calibration: Efficient and confident test-time scaling

arXiv: https://arxiv.org/abs/2505.18404 · HTML: https://mlsanity.com/paper/2505.18404/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 20 of its 51 references that are in the map.
- First posted on arXiv 23 May 2025.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 20 of 20)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [s1: Simple test-time scaling](https://mlsanity.com/paper/2501.19393/) (2025)
- [A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification](https://mlsanity.com/paper/2107.07511/) (2021)
- [Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs](https://mlsanity.com/paper/2412.21187/) (2024)
- [Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models](https://mlsanity.com/paper/2503.16419/) (2025)
- [Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Control](https://mlsanity.com/paper/2110.01052/) (2021)
- [Token-Budget-Aware LLM Reasoning](https://mlsanity.com/paper/2412.18547/) (2024)
- [Training Language Models to Reason Efficiently](https://mlsanity.com/paper/2502.04463/) (2025)
- [Dynamic Early Exit in Reasoning Models](https://mlsanity.com/paper/2504.15895/) (2025)
- [ThinkPrune: Pruning Long Chain-of-Thought of LLMs via Reinforcement Learning](https://mlsanity.com/paper/2504.01296/) (2025)
- [Reasoning Models Can Be Effective Without Thinking](https://mlsanity.com/paper/2504.09858/) (2025)
- [Reasoning Models Know When They're Right: Probing Hidden States for Self-Verification](https://mlsanity.com/paper/2504.05419/) (2025)
- [Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMa-3 8B](https://mlsanity.com/paper/2406.07394/) (2024)
- [Efficient Test-Time Scaling via Self-Calibration](https://mlsanity.com/paper/2503.00031/) (2025)
- [Calibrating Reasoning in Language Models with Internal Consistency](https://mlsanity.com/paper/2405.18711/) (2024)
- [Temporal Consistency for LLM Reasoning Process Error Identification](https://mlsanity.com/paper/2503.14495/) (2025)

## Built on by (most cited first, 21 of 21)

- [Dynamic Model Routing and Cascading for Efficient LLM Inference: A Survey](https://mlsanity.com/paper/2603.04445/) (2026)
- [E-valuator: Reliable Agent Verifiers with Sequential Hypothesis Testing](https://mlsanity.com/paper/2512.03109/) (2025)
- [Conformal Thinking: Risk Control for Reasoning on a Compute Budget](https://mlsanity.com/paper/2602.03814/) (2026)
- [Entropy trajectory shape predicts LLM reasoning reliability: A diagnostic study of uncertainty dynamics in chain-of-thought](https://mlsanity.com/paper/2603.18940/) (2026)
- [Statistical Early Stopping for Reasoning Models](https://mlsanity.com/paper/2602.13935/) (2026)
- [ReJump: A Tree-Jump Representation for Analyzing and Improving LLM Reasoning](https://mlsanity.com/paper/2512.00831/) (2025)
- [From Reasoning Traces to Reusable Modules: Understanding Compositional Generalization in Language Model Reasoning](https://mlsanity.com/paper/2606.18089/) (2026)
- [Online Reasoning Calibration: Test-Time Training Enables Generalizable Conformal LLM Reasoning](https://mlsanity.com/paper/2604.01170/) (2026)
- [When Should an AI Workflow Release? Always-Valid Inference for Black-Box Generate-Verify Systems](https://mlsanity.com/paper/2605.12947/) (2026)
- [Reliable LLM-Based Edge-Cloud-Expert Cascades for Telecom Knowledge Systems](https://mlsanity.com/paper/2512.20012/) (2025)
- [Process Supervision of Confidence Margin for Calibrated LLM Reasoning](https://mlsanity.com/paper/2604.23333/) (2026)
- [SHERLOC: Structured Diagnostic Localization for Code Repair Agents](https://mlsanity.com/paper/2606.24820/) (2026)
- [When Does Learning to Stop Help? A Cost-Aware Study of Early Exits in Reasoning Models](https://mlsanity.com/paper/2606.30852/) (2026)
- [Think Short, Defer Smart, Act, and Repeat: Calibrated Reasoning and Uncertainty-Aware Deferral for Edge LLM Agents](https://mlsanity.com/paper/2607.26865/) (2026)
- [Prefix Sliding for efficient test-time scaling](https://mlsanity.com/paper/2608.26070/) (2026)
- [Conformal Language Modeling via Posterior Sampling](https://mlsanity.com/paper/2606.03731/) (2026)
- [Algebraic Quantum Intelligence: A New Framework for Reproducible Machine Creativity](https://mlsanity.com/paper/2602.14130/) (2026)
- [Primal Generation, Dual Judgment: Self-Training from Test-Time Scaling](https://mlsanity.com/paper/2605.11299/) (2026)
- [Online Safety Monitoring for LLMs](https://mlsanity.com/paper/2607.02510/) (2026)
- [Code-MUE: Measuring Code LLMs' Uncertainty through Execution-based Semantic Interaction Graphs](https://mlsanity.com/paper/2607.12273/) (2026)
- [Funnel of Thoughts: Efficient Test-Time Scaling via Early Voting and Rollout Pruning](https://mlsanity.com/paper/2608.15065/) (2026)

## Abstract (arXiv)

Reasoning large language models achieve impressive test-time scaling by thinking for longer, but this performance gain comes at significant compute cost. Directly limiting test-time budget hurts overall performance, but not all problems are equally difficult. We propose thought calibration to decide dynamically when thinking can be terminated. To calibrate our decision rule, we view a language model's growing body of thoughts as a nested sequence of reasoning trees, where the goal is to identify the point at which novel reasoning plateaus. We realize this framework through lightweight probes that operate on top of the language model's hidden representations, which are informative of both the reasoning structure and overall consistency of response. Based on three reasoning language models and four datasets, thought calibration preserves model performance with up to a 60% reduction in thinking tokens on in-distribution data, and up to 20% in out-of-distribution data.