# Fact-Level Confidence Calibration and Self-Correction

arXiv: https://arxiv.org/abs/2411.13343 · HTML: https://mlsanity.com/paper/2411.13343/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Nov 2024.

## Built on by (most cited first, 5 of 5)

- [Incentivizing Strong Reasoning from Weak Supervision](https://mlsanity.com/paper/2505.20072/) (2025)
- [Goal-Aware Identification and Rectification of Misinformation in Multi-Agent Systems](https://mlsanity.com/paper/2506.00509/) (2025)
- [Transparentize the Internal and External Knowledge Utilization in LLMs with Trustworthy Citation](https://mlsanity.com/paper/2504.14856/) (2025)
- [Localized Calibrated Uncertainty in Code Language Models](https://mlsanity.com/paper/2512.24560/) (2025)
- [LLMs Know When They Know, but Do Not Act on It: A Metacognitive Harness for Test-time Scaling](https://mlsanity.com/paper/2605.14186/) (2026)

## Abstract (arXiv)

Confidence calibration in LLMs, i.e., aligning their self-assessed confidence with the actual accuracy of their responses, enabling them to self-evaluate the correctness of their outputs. However, current calibration methods for LLMs typically estimate two scalars to represent overall response confidence and correctness, which is inadequate for long-form generation where the response includes multiple atomic facts and may be partially confident and correct. These methods also overlook the relevance of each fact to the query. To address these challenges, we propose a Fact-Level Calibration framework that operates at a finer granularity, calibrating confidence to relevance-weighted correctness at the fact level. Furthermore, comprehensive analysis under the framework inspired the development of Confidence-Guided Fact-level Self-Correction (ConFix), which uses high-confidence facts within a response as additional knowledge to improve low-confidence ones. Extensive experiments across four datasets and six models demonstrate that ConFix effectively mitigates hallucinations without requiring external knowledge sources such as retrieval systems.