The Missing "I Don't Know": Why Three Reasoning-Reliability Findings Converge on Calibrated Abstention
On the map: LLM reasoning → Test-time compute and efficient thinking
Abstract (arXiv)
Three recent results describe what look like unrelated LLM reliability problems. Yin et al. (2026) show reasoning RL collapses tool-reliability representations. Suleymanov et al. (2026) show that under safety-constrained generation, large models rewrite flagged spans while small models truncate. Bastounis et al. (2024) prove any consistent-reasoning system without an implicit "I don't know" function must hallucinate infinitely often on broad problem classes. We argue these findings converge on a single intervention: calibrated abstention is what each independently identifies as the missing capability, even though the unavailability they document, a capability gap, a policy gap, and a recursion-theoretic gap, has a different source in each case. Honesty post-training has narrowed the gap in deployed models, but principled closure of the class Bastounis identifies requires a calibrated abstention function whose training signal at the leaderboard level is absent: dominant benchmarks assign zero reward to decline, so the leaderboard gradient that would select for the function does not exist. We propose four changes to evaluation: triple-scoring, abstention-rate reporting, capability-stratified evaluation, and mandatory calibration metrics. Benchmark reform is necessary, not sufficient, for closing the gap the theorem identifies.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 17 of its 24 references that are in the map. First posted on arXiv 15 Sep 2026.
Builds on
17 of 24 references.
Closest ancestors · 1
Same direction, newest first
Other references · 16
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2020Measuring Massive Multitask Language Understandingcited by 7,963
- 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
- 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
- 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
- 2017Selective Classification for Deep Neural Networkscited by 852
- 2024Measuring short-form factuality in large language modelscited by 373
- 2019Verified Uncertainty Calibrationcited by 309
- 2020Selective Question Answering under Domain Shiftcited by 238
- 2024Are We Done with MMLU?cited by 194
- 2025AbstentionBench: Reasoning LLMs Fail on Unanswerable Questionscited by 145
4 more
- 2020Mitigating Bias in Calibration Error Estimationcited by 101
- 2025The Reasoning Trap: How Enhancing LLM Reasoning Amplifies Tool Hallucinationcited by 14
- 2025Honesty over Accuracy: Trustworthy Language Models through Reinforced Hesitationcited by 5
- 2026Beyond Refusal: Probing the Limits of Agentic Self-Correction for Semantic Sensitive Informationcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Quantifying Faithful Confidence Expression in Large Reasoning Models
- 2025Reasoning about Uncertainty: Do Reasoning Models Know When They Don't Know?
- 2026Amplified Does Not Mean Predictive: Reasoning Behaviors in Thinking Models
- 2026Knowing When to Quit: Diagnosing and Training LLMs to Abort Futile Reasoning
- 2026Calibration Drift Under Reasoning: How Chain-of-Thought Budgets Induce Overconfidence in Large Language Models
- 2026Bridging the Detection-to-Abstention Gap in Reasoning Models under Insufficient Information
- 2025Reasoning Models Better Express Their Confidence
- 2026Knowing When to Quit: A Principled Framework for Dynamic Abstention in LLM Reasoning