# Conformal Thinking: Risk Control for Reasoning on a Compute Budget

arXiv: https://arxiv.org/abs/2602.03814 · HTML: https://mlsanity.com/paper/2602.03814/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 24 of its 29 references that are in the map.
- First posted on arXiv 3 Feb 2026.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 24 of 24)

- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset](https://mlsanity.com/paper/2402.14804/) (2024)
- [Humanity's Last Exam](https://mlsanity.com/paper/2501.14249/) (2025)
- [Confident Adaptive Language Modeling](https://mlsanity.com/paper/2207.07061/) (2022)
- [Conformal Risk Control](https://mlsanity.com/paper/2208.02814/) (2022)
- [Distribution-Free, Risk-Controlling Prediction Sets](https://mlsanity.com/paper/2101.02703/) (2021)
- [Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Control](https://mlsanity.com/paper/2110.01052/) (2021)
- [Dynamic Early Exit in Reasoning Models](https://mlsanity.com/paper/2504.15895/) (2025)
- [AIMO-2 Winning Solution: Building State-of-the-Art Mathematical Reasoning Models with OpenMathReasoning dataset](https://mlsanity.com/paper/2504.16891/) (2025)
- [Reasoning Models Know When They're Right: Probing Hidden States for Self-Verification](https://mlsanity.com/paper/2504.05419/) (2025)
- [Efficiently Scaling LLM Reasoning with Certaindex](https://mlsanity.com/paper/2412.20993/) (2024)
- [Answer Convergence as a Signal for Early Stopping in Reasoning](https://mlsanity.com/paper/2506.02536/) (2025)
- [Fast yet Safe: Early-Exiting with Risk Control](https://mlsanity.com/paper/2405.20915/) (2024)
- [Entropy After </Think> for reasoning model early exiting](https://mlsanity.com/paper/2509.26522/) (2025)
- [Thought calibration: Efficient and confident test-time scaling](https://mlsanity.com/paper/2505.18404/) (2025)
- [Early Stopping Chain-of-thoughts in Large Language Models](https://mlsanity.com/paper/2509.14004/) (2025)
- [The Evolution of Thought: Tracking LLM Overthinking via Reasoning Dynamics Analysis](https://mlsanity.com/paper/2508.17627/) (2025)
- [Is That Your Final Answer? Test-Time Scaling Improves Selective Question Answering](https://mlsanity.com/paper/2502.13962/) (2025)
- [E-valuator: Reliable Agent Verifiers with Sequential Hypothesis Testing](https://mlsanity.com/paper/2512.03109/) (2025)
- [Adaptive Deep Reasoning: Triggering Deep Thinking When Needed](https://mlsanity.com/paper/2505.20101/) (2025)
- [Early Time Classification with Accumulated Accuracy Gap Control](https://mlsanity.com/paper/2402.00857/) (2024)
- [On the Provable Performance Guarantee of Efficient Reasoning Models](https://mlsanity.com/paper/2510.09133/) (2025)
- [DTS: Enhancing Large Reasoning Models via Decoding Tree Sketching](https://mlsanity.com/paper/2511.00640/) (2025)

## Built on by (most cited first, 10 of 10)

- [Early Stopping for Large Reasoning Models via Confidence Dynamics](https://mlsanity.com/paper/2604.04930/) (2026)
- [Online Reasoning Calibration: Test-Time Training Enables Generalizable Conformal LLM Reasoning](https://mlsanity.com/paper/2604.01170/) (2026)
- [Unified Multimodal Uncertain Inference](https://mlsanity.com/paper/2604.08701/) (2026)
- [Process Supervision of Confidence Margin for Calibrated LLM Reasoning](https://mlsanity.com/paper/2604.23333/) (2026)
- [When Does Learning to Stop Help? A Cost-Aware Study of Early Exits in Reasoning Models](https://mlsanity.com/paper/2606.30852/) (2026)
- [Revelation Control](https://mlsanity.com/paper/2608.23860/) (2026)
- [Safe to Stop? Risk-Constrained Stopping for Sequential Clinical Diagnosis Agents](https://mlsanity.com/paper/2609.09678/) (2026)
- [DAR: Deontic Reasoning with Agentic Harnesses](https://mlsanity.com/paper/2606.05009/) (2026)
- [Online Safety Monitoring for LLMs](https://mlsanity.com/paper/2607.02510/) (2026)
- [Adaptive Computation with Conformal Certification](https://mlsanity.com/paper/2610.19189/) (2026)

## Abstract (arXiv)

Reasoning Large Language Models (LLMs) enable test-time scaling, with dataset-level accuracy improving as the token budget increases, motivating adaptive reasoning -- spending tokens when they improve reliability and stopping early when additional computation is unlikely to help. However, setting the token budget, as well as the threshold for adaptive reasoning, is a practical challenge that entails a fundamental risk-accuracy trade-off. We re-frame the budget setting problem as risk control, limiting the error rate while minimizing compute. Our framework introduces an upper threshold that stops reasoning when the model is confident (risking incorrect output) and a novel parametric lower threshold that preemptively stops unsolvable instances (risking premature stoppage). Given a target risk and a validation set, we use distribution-free risk control to optimally specify these stopping mechanisms. For scenarios with multiple budget controlling criteria, we incorporate an efficiency loss to select the most computationally efficient exiting mechanism. Empirical results across diverse reasoning tasks and models demonstrate the effectiveness of our risk control approach, demonstrating computational efficiency gains from the lower threshold and ensemble stopping mechanisms while adhering to the user-specified risk target.