Are LLM Decisions Faithful to Verbal Confidence?
Not on the map. Why?
Abstract (arXiv)
Large Language Models (LLMs) can produce surprisingly sophisticated estimates of their own uncertainty. However, it remains unclear to what extent this expressed confidence is tied to the reasoning, knowledge, or decision making of the model. To test this, we introduce RiskEval: a framework designed to evaluate whether models adjust their abstention policies in response to varying error penalties. Our evaluation of several frontier models reveals a critical dissociation: models are neither cost-aware when articulating their verbal confidence, nor strategically responsive when deciding whether to engage or abstain under high-penalty conditions. Even when extreme penalties render frequent abstention the mathematically optimal strategy, models almost never abstain, resulting in utility collapse. This indicates that calibrated verbal confidence scores may not be sufficient to create trustworthy and interpretable AI systems, as current models lack the strategic agency to convert uncertainty signals into optimal and risk-sensitive decisions.
Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 20 of its 36 references that are in the map. First posted on arXiv 12 Jan 2026.
Builds on
20 of 36 references.
References · 20
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
- 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
- 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 788
- 2022Teaching Models to Express Their Uncertainty in Wordscited by 731
- 2023A Survey of Confidence Estimation and Calibration in Large Language Modelscited by 253
- 2025AbstentionBench: Reasoning LLMs Fail on Unanswerable Questionscited by 129
- 2025Beyond Binary Rewards: Training LMs to Reason About Their Uncertaintycited by 107
- 2022DisentQA: Disentangling Parametric and Contextual Knowledge with Counterfactual Question Answeringcited by 93
- 2025Reasoning Models Better Express Their Confidencecited by 63
- 2024Calibrating the Confidence of Large Language Models by Eliciting Fidelitycited by 47
8 more
- 2024LLM economicus? Mapping the Behavioral Biases of LLMs via Utility Theorycited by 44
- 2024Decision-Making Behavior Evaluation Framework for LLMs under Uncertain Contextcited by 41
- 2024Atomic Calibration of LLMs in Long-Form Generationscited by 23
- 2025From Calibration to Collaboration: LLM Uncertainty Quantification Should Be More Human-Centeredcited by 21
- 2025LoVeC: Reinforcement Learning for Better Verbalized Confidence in Long-Form Generationscited by 17
- 2025Trace Length is a Simple Uncertainty Signal in Reasoning Modelscited by 16
- 2024Teaching LLMs to Abstain across Languages via Multilingual Feedbackcited by 14
- 2025Evaluating and Aligning Human Economic Risk Preferences in LLMscited by 7
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026CoT2-Meta: Budgeted Metacognitive Control for Test-Time Reasoningcited by 2
- 2026Reflective Prompt Tuning through Language Model Function-Callingcited by 1
- 2026When Policies Change Probabilities: Modular Decision-Making for LLM Code Reviewcited by 1
- 2026Pander Score: A Continuous Measure of Sycophancy as Epistemic Deferencecited by 0
- 2026Stable Miscalibration in Large Language Models: A Practical View of High-Confidence Errorscited by 0
- 2026A Tale of Two Error Categories: Exploring Concealed Trade-Offs in the Errors of Automated Judges in Evaluation of Uncertainty Quantifierscited by 0
- 2026A Tale of Two Error Categories: Exploring Concealed Trade-Offs in the Errors of Automated Judges in Evaluation of Uncertainty Quantifiers7 Oct 2026
- 2026When Policies Change Probabilities: Modular Decision-Making for LLM Code Review2 Aug 2026
- 2026Stable Miscalibration in Large Language Models: A Practical View of High-Confidence Errors15 Jul 2026
- 2026Pander Score: A Continuous Measure of Sycophancy as Epistemic Deference5 Jun 2026
- 2026Reflective Prompt Tuning through Language Model Function-Calling20 May 2026
- 2026CoT2-Meta: Budgeted Metacognitive Control for Test-Time Reasoning30 Mar 2026