# Learning to Stop without Learning to Stop: Self-Supervised Confidence Training Improves Reasoning Efficiency

arXiv: https://arxiv.org/abs/2609.31619 · HTML: https://mlsanity.com/paper/2609.31619/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 34 of its 37 references that are in the map.
- First posted on arXiv 25 Sep 2026.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 30 of 34)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning](https://mlsanity.com/paper/2503.04697/) (2025)
- [DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoning](https://mlsanity.com/paper/2504.11456/) (2025)
- [Training Language Models to Reason Efficiently](https://mlsanity.com/paper/2502.04463/) (2025)
- [Dynamic Early Exit in Reasoning Models](https://mlsanity.com/paper/2504.15895/) (2025)
- [ThinkPrune: Pruning Long Chain-of-Thought of LLMs via Reinforcement Learning](https://mlsanity.com/paper/2504.01296/) (2025)
- [Reasoning Models Know When They're Right: Probing Hidden States for Self-Verification](https://mlsanity.com/paper/2504.05419/) (2025)
- [Beyond Benchmarks: MathArena as an Evaluation Platform for Mathematics with LLMs](https://mlsanity.com/paper/2605.00674/) (2026)
- [Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty](https://mlsanity.com/paper/2507.16806/) (2025)
- [Self-Training Elicits Concise Reasoning in Large Language Models](https://mlsanity.com/paper/2502.20122/) (2025)
- [ShorterBetter: Guiding Reasoning Models to Find Optimal Inference Length for Efficient Reasoning](https://mlsanity.com/paper/2504.21370/) (2025)
- [Rewarding Doubt: A Reinforcement Learning Approach to Calibrated Confidence Expression of Large Language Models](https://mlsanity.com/paper/2503.02623/) (2025)
- [Answer Convergence as a Signal for Early Stopping in Reasoning](https://mlsanity.com/paper/2506.02536/) (2025)
- [DLER: Doing Length pEnalty Right - Incentivizing More Intelligence per Token via Reinforcement Learning](https://mlsanity.com/paper/2510.15110/) (2025)
- [Think or Not? Exploring Thinking Efficiency in Large Reasoning Models via an Information-Theoretic Lens](https://mlsanity.com/paper/2505.18237/) (2025)
- [ConCISE: Confidence-guided Compression in Step-by-step Efficient Reasoning](https://mlsanity.com/paper/2505.04881/) (2025)
- [Entropy After </Think> for reasoning model early exiting](https://mlsanity.com/paper/2509.26522/) (2025)
- [Early Stopping Chain-of-thoughts in Large Language Models](https://mlsanity.com/paper/2509.14004/) (2025)
- [Does Your Reasoning Model Implicitly Know When to Stop Thinking?](https://mlsanity.com/paper/2602.08354/) (2026)
- [Your Models Have Thought Enough: Training Large Reasoning Models to Stop Overthinking](https://mlsanity.com/paper/2509.23392/) (2025)
- [Verbalized Confidence Triggers Self-Verification: Emergent Behavior Without Explicit Reasoning Supervision](https://mlsanity.com/paper/2506.03723/) (2025)
- [Understanding the Thinking Process of Reasoning Models: A Perspective from Schoenfeld's Episode Theory](https://mlsanity.com/paper/2509.14662/) (2025)
- [Early Stopping for Large Reasoning Models via Confidence Dynamics](https://mlsanity.com/paper/2604.04930/) (2026)
- [Understanding and Mitigating Premature Confidence for Better LLM Reasoning](https://mlsanity.com/paper/2605.24396/) (2026)
- [On-Policy Supervised Fine-Tuning for Efficient Reasoning](https://mlsanity.com/paper/2602.13407/) (2026)
- [Reinforcement Learning with Metacognitive Feedback Elicits Faithful Uncertainty Expression in LLMs](https://mlsanity.com/paper/2606.32032/) (2026)

## Abstract (arXiv)

Reasoning models often generate very long reasoning traces, making inference computationally expensive. Existing approaches typically improve efficiency either through inference-time early-stopping mechanisms or by explicitly encouraging shorter reasoning during training, for example through reinforcement learning with length penalties. We show that substantial efficiency gains can instead emerge from a different kind of supervision: textitconfidence. Using a self-supervised procedure, we fine-tune reasoning models to predict their confidence in the answer at intermediate points along their own reasoning trajectories using only 600 training problems. Confidence is used only as a training target: the loss contains no objective for reasoning length, efficiency, or stopping. At inference, the fine-tuned models use the standard generation procedure, with no confidence elicitation or early-stopping mechanism. Despite this, self-supervised confidence fine-tuning makes reasoning more efficient, reducing generated tokens by up to 25% at matched accuracy across Gemma, Qwen, Nemotron, and GPT-OSS models on mathematical, scientific, and coding reasoning benchmarks, with efficiency gains comparable to methods that explicitly optimize for shorter reasoning. Analysis of reasoning episodes further shows that confidence supervision largely preserves the base models' high-level reasoning composition rather than selectively suppressing particular behaviors. Our results suggest that efficient reasoning may emerge as a downstream consequence of learning metacognitive signals, without being directly optimized.