# Teaching LLMs to Abstain via Fine-Grained Semantic Confidence Reward

arXiv: https://arxiv.org/abs/2510.24020 · HTML: https://mlsanity.com/paper/2510.24020/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Oct 2025.

## Built on by (most cited first, 4 of 4)

- [I-CALM: Incentivizing Confidence-Aware Abstention for LLM Selective Answering](https://mlsanity.com/paper/2604.03904/) (2026)
- [Purging the Gray Zone: Latent-Geometric Denoising for Precise Knowledge Boundary Awareness](https://mlsanity.com/paper/2604.14324/) (2026)
- [Only Say What You Know: Calibration-Aware Generation for Long-Form Factuality](https://mlsanity.com/paper/2605.01749/) (2026)
- [Stable Miscalibration in Large Language Models: A Practical View of High-Confidence Errors](https://mlsanity.com/paper/2608.13591/) (2026)

## Abstract (arXiv)

Mitigating hallucinations in Large Language Models (LLMs) is critical for their reliable deployment. Existing methods typically fine-tune LLMs to abstain from answering questions beyond their knowledge scope. However, these methods often rely on coarse-grained signals to guide LLMs to abstain, such as overall confidence or uncertainty scores on multiple sampled answers, which may result in an imprecise awareness of the model's own knowledge boundaries. To this end, we propose a novel reinforcement learning framework built on textbfunderlineFine-grained underlineSemantic underlineConfidence underlineReward (Ours), which guides LLMs to abstain via sample-specific confidence. Specifically, our method operates by sampling multiple candidate answers and conducting semantic clustering, then training the LLM to retain answers within high-confidence clusters and discard those within low-confidence ones, thereby promoting accurate post-hoc abstention. Additionally, we propose a new metric for evaluating the reliability of abstention fine-tuning tasks more comprehensively. Our method significantly enhances reliability in both in-domain and out-of-distribution benchmarks.