# Reported Confidence in LLMs Tracks Commitment More Than Correctness

arXiv: https://arxiv.org/abs/2606.29490 · HTML: https://mlsanity.com/paper/2606.29490/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 39 references that are in the map.
- First posted on arXiv 28 Jun 2026.

## Builds on (most cited first, 13 of 13)

- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs](https://mlsanity.com/paper/2306.13063/) (2023)
- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback](https://mlsanity.com/paper/2305.14975/) (2023)
- [Teaching Models to Express Their Uncertainty in Words](https://mlsanity.com/paper/2205.14334/) (2022)
- [Measuring short-form factuality in large language models](https://mlsanity.com/paper/2411.04368/) (2024)
- [Deep Think with Confidence](https://mlsanity.com/paper/2508.15260/) (2025)
- [Emergent Introspective Awareness in Large Language Models](https://mlsanity.com/paper/2601.01828/) (2026)
- [SuperGPQA: Scaling LLM Evaluation across 285 Graduate Disciplines](https://mlsanity.com/paper/2502.14739/) (2025)
- [Reasoning Models Better Express Their Confidence](https://mlsanity.com/paper/2505.14489/) (2025)
- [How do LLMs Compute Verbal Confidence](https://mlsanity.com/paper/2603.17839/) (2026)
- [Causal Evidence that Language Models use Confidence to Drive Behavior](https://mlsanity.com/paper/2603.22161/) (2026)
- [Mechanisms of Introspective Awareness](https://mlsanity.com/paper/2603.21396/) (2026)

## Built on by (most cited first, 8 of 8)

- [Abstention as an Action Can Kill Both the Reward Gradient and the KL Anchor: Collapse Law and Repair for Error-Penalized Reinforcement Learning](https://mlsanity.com/paper/2608.00301/) (2026)
- [When Self-Consistency Backfires: Majority Vote Hurts the Majority of Hard Science Problems for Small LLMs](https://mlsanity.com/paper/2608.11403/) (2026)
- [The Computational Basis of Confidence in Large Language Models](https://mlsanity.com/paper/2607.12447/) (2026)
- [Uncertainty Quantification for LLM Agents: A Taxonomy, an Evaluation Protocol, and an Empirical Study](https://mlsanity.com/paper/2609.07395/) (2026)
- [Confidence Comes from Experience: Experiential Confidence Estimation from Reasoning to Agents](https://mlsanity.com/paper/2609.17708/) (2026)
- [Reasoning Concentrates Errors, and Self-Consistency Never Notices](https://mlsanity.com/paper/2609.32035/) (2026)
- [Calibration, Not Answer Selection: Distilling Internal Confidence in Reasoning Models](https://mlsanity.com/paper/2609.33290/) (2026)
- [LLMs learn different forms of metacognition when trained to predict their own accuracy](https://mlsanity.com/paper/2609.33886/) (2026)

## Abstract (arXiv)

Confidence is an estimate of the probability that a chosen answer is correct. Verbal confidence reports are widely used as uncertainty measures in large language models, but whether they are best understood as estimates of correctness is unclear. We test this with a two-stage abstention paradigm from the neuroscience of perceptual decision making: a model first answers and reports its confidence, then decides whether to commit it to a user or abstain. Across four non-reasoning models, prompt framings, and confidence formats, verbal confidence predicted the commit/abstain decision substantially better than whether the answer was correct. Calibrated token log-probabilities showed the opposite profile, with abstention-prediction coupled to correctness discrimination, the signature of an answer-evidence signal. After removing the variance verbal confidence shared with log-probabilities, the residual stayed aligned with commitment while its link to correctness fell to near chance. The dissociation generalised to four reasoning models across four benchmarks of varying difficulty, from hard multiple-choice to frontier-level freeform questions. Mechanistic analyses in Gemma 3 and 4 were convergent: a post-answer state known to causally support verbal-confidence generation already encoded the future abstention decision before the abstention prompt, organised mainly by that decision rather than by correctness, the two lying in approximately orthogonal directions in activation space. Steering along a verbal-confidence-specific direction causally shifted abstention. Verbal and log-probability confidence are thus not interchangeable: log-probabilities track answer evidence and correctness, whereas verbal confidence is better understood as a behaviour-facing readout of an internal commit-readiness state, challenging the practice of treating verbal reports as proxies for reliability.