Reported Confidence in LLMs Tracks Commitment More Than Correctness
Not on the map. Why?
Abstract (arXiv)
Confidence is an estimate of the probability that a chosen answer is correct. Verbal confidence reports are widely used as uncertainty measures in large language models, but whether they are best understood as estimates of correctness is unclear. We test this with a two-stage abstention paradigm from the neuroscience of perceptual decision making: a model first answers and reports its confidence, then decides whether to commit it to a user or abstain. Across four non-reasoning models, prompt framings, and confidence formats, verbal confidence predicted the commit/abstain decision substantially better than whether the answer was correct. Calibrated token log-probabilities showed the opposite profile, with abstention-prediction coupled to correctness discrimination, the signature of an answer-evidence signal. After removing the variance verbal confidence shared with log-probabilities, the residual stayed aligned with commitment while its link to correctness fell to near chance. The dissociation generalised to four reasoning models across four benchmarks of varying difficulty, from hard multiple-choice to frontier-level freeform questions. Mechanistic analyses in Gemma 3 and 4 were convergent: a post-answer state known to causally support verbal-confidence generation already encoded the future abstention decision before the abstention prompt, organised mainly by that decision rather than by correctness, the two lying in approximately orthogonal directions in activation space. Steering along a verbal-confidence-specific direction causally shifted abstention. Verbal and log-probability confidence are thus not interchangeable: log-probabilities track answer evidence and correctness, whereas verbal confidence is better understood as a behaviour-facing readout of an internal commit-readiness state, challenging the practice of treating verbal reports as proxies for reliability.
Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 13 of its 39 references that are in the map. First posted on arXiv 28 Jun 2026.
Builds on
13 of 39 references.
References · 13
- 2022Language Models (Mostly) Know What They Knowcited by 1,856
- 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
- 2023Steering Language Models With Activation Engineeringcited by 922
- 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 812
- 2022Teaching Models to Express Their Uncertainty in Wordscited by 751
- 2024Measuring short-form factuality in large language modelscited by 373
- 2025Deep Think with Confidencecited by 205
- 2026Emergent Introspective Awareness in Large Language Modelscited by 82
- 2025SuperGPQA: Scaling LLM Evaluation across 285 Graduate Disciplinescited by 79
- 2025Reasoning Models Better Express Their Confidencecited by 65
- 2026How do LLMs Compute Verbal Confidencecited by 26
- 2026Causal Evidence that Language Models use Confidence to Drive Behaviorcited by 10
1 more
- 2026Mechanisms of Introspective Awarenesscited by 7
Built on by
Citing papers per year: 2026: 8.
8 papers.
- 2026Abstention as an Action Can Kill Both the Reward Gradient and the KL Anchor: Collapse Law and Repair for Error-Penalized Reinforcement Learningcited by 3
- 2026When Self-Consistency Backfires: Majority Vote Hurts the Majority of Hard Science Problems for Small LLMscited by 3
- 2026The Computational Basis of Confidence in Large Language Modelscited by 1
- 2026Uncertainty Quantification for LLM Agents: A Taxonomy, an Evaluation Protocol, and an Empirical Studycited by 0
- 2026Confidence Comes from Experience: Experiential Confidence Estimation from Reasoning to Agentscited by 0
- 2026Reasoning Concentrates Errors, and Self-Consistency Never Noticescited by 0
- 2026Calibration, Not Answer Selection: Distilling Internal Confidence in Reasoning Modelscited by 0
- 2026LLMs learn different forms of metacognition when trained to predict their own accuracycited by 0
- 2026LLMs learn different forms of metacognition when trained to predict their own accuracy27 Sep 2026
- 2026Calibration, Not Answer Selection: Distilling Internal Confidence in Reasoning Models27 Sep 2026
- 2026Reasoning Concentrates Errors, and Self-Consistency Never Notices25 Sep 2026
- 2026Confidence Comes from Experience: Experiential Confidence Estimation from Reasoning to Agents15 Sep 2026
- 2026Uncertainty Quantification for LLM Agents: A Taxonomy, an Evaluation Protocol, and an Empirical Study7 Sep 2026
- 2026When Self-Consistency Backfires: Majority Vote Hurts the Majority of Hard Science Problems for Small LLMs11 Aug 2026
- 2026Abstention as an Action Can Kill Both the Reward Gradient and the KL Anchor: Collapse Law and Repair for Error-Penalized Reinforcement Learning31 Jul 2026
- 2026The Computational Basis of Confidence in Large Language Models14 Jul 2026
Similar, not cited
Close in wording, no citation either way
- 2024Confidence in the Reasoning of Large Language Models
- 2026An Analysis of Training-Free Self-Reported Confidence in Language Models
- 2026LLMs Should Express Uncertainty Explicitly
- 2026Certainty Is Not Just Correctness: Rethinking Token-Level Certainty in LLM Reasoning
- 2026CALIBER: Calibrating Confidence Before and After Reasoning in Language Models
- 2024Confidence Under the Hood: An Investigation into the Confidence-Probability Alignment in Large Language Models
- 2025Let the Model Distribute Its Doubt: Confidence Estimation through Verbalized Probability Distribution
- 2026Understanding and Mitigating Premature Confidence for Better LLM Reasoning