# Stable Miscalibration in Large Language Models: A Practical View of High-Confidence Errors

arXiv: https://arxiv.org/abs/2608.13591 · HTML: https://mlsanity.com/paper/2608.13591/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 16 references that are in the map.
- First posted on arXiv 15 Jul 2026.

## Builds on (most cited first, 10 of 10)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models](https://mlsanity.com/paper/2303.08896/) (2023)
- [Know Your Limits: A Survey of Abstention in Large Language Models](https://mlsanity.com/paper/2407.18418/) (2024)
- [LLM Internal States Reveal Hallucination Risk Faced With a Query](https://mlsanity.com/paper/2407.03282/) (2024)
- [Restoring Calibration for Aligned Large Language Models: A Calibration-Aware Fine-Tuning Approach](https://mlsanity.com/paper/2505.01997/) (2025)
- [Learning Conformal Abstention Policies for Adaptive Risk Management in Large Language and Vision-Language Models](https://mlsanity.com/paper/2502.06884/) (2025)
- [Are LLM Decisions Faithful to Verbal Confidence?](https://mlsanity.com/paper/2601.07767/) (2026)
- [Black-Box Hallucination Detection via Consistency Under the Uncertain Expression](https://mlsanity.com/paper/2509.21999/) (2025)
- [Teaching LLMs to Abstain via Fine-Grained Semantic Confidence Reward](https://mlsanity.com/paper/2510.24020/) (2025)

## Abstract (arXiv)

High-confidence errors in large language models are often treated as evidence of fragile internal inference. We study a different possibility: stable miscalibration, where a confident wrong answer remains locally stable under small perturbations. We combine two diagnostics: a label-aware output-level audit score that ranks domains by confidence variation and overconfident mistakes under a forced-answer baseline, and an internal sensitivity probe that measures hidden-state movement. On a multi-domain binary factual audit set, this audit score tracks where abstention-aware self-critique reduces decision loss, although direct labeled baselines rank the same gain more strongly. Internally, self-critical prompting consistently reduces hidden-state sensitivity across layers in three open-weight models. This supports prompt-induced local stabilization rather than a purely output-level abstention pattern, but it does not imply calibration: audit-defined overconfident errors are not clearly more locally sensitive than confidently correct answers, so some high-confidence errors may be stable and miscalibrated rather than simply fragile.