# I-CALM: Incentivizing Confidence-Aware Abstention for LLM Selective Answering

arXiv: https://arxiv.org/abs/2604.03904 · HTML: https://mlsanity.com/paper/2604.03904/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 81 references that are in the map.
- First posted on arXiv 5 Apr 2026.

## Builds on (most cited first, 30 of 38)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs](https://mlsanity.com/paper/2306.13063/) (2023)
- [Teaching Models to Express Their Uncertainty in Words](https://mlsanity.com/paper/2205.14334/) (2022)
- [Measuring short-form factuality in large language models](https://mlsanity.com/paper/2411.04368/) (2024)
- [Why Language Models Hallucinate](https://mlsanity.com/paper/2509.04664/) (2025)
- [AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions](https://mlsanity.com/paper/2506.09038/) (2025)
- [Language Model Cascades: Token-level uncertainty and beyond](https://mlsanity.com/paper/2404.10136/) (2024)
- [Taming Overconfidence in LLMs: Reward Calibration in RLHF](https://mlsanity.com/paper/2410.09724/) (2024)
- ["My Answer is C": First-Token Probabilities Do Not Match Text Answers in Instruction-Tuned Language Models](https://mlsanity.com/paper/2402.14499/) (2024)
- [A Survey of Uncertainty Estimation Methods on Large Language Models](https://mlsanity.com/paper/2503.00172/) (2025)
- [Uncertainty-Based Abstention in LLMs Improves Safety and Reduces Hallucinations](https://mlsanity.com/paper/2404.10960/) (2024)
- [SimpleQA Verified: A Reliable Factuality Benchmark to Measure Parametric Knowledge](https://mlsanity.com/paper/2509.07968/) (2025)
- [Can AI Assistants Know What They Don't Know?](https://mlsanity.com/paper/2401.13275/) (2024)
- [Mind the Confidence Gap: Overconfidence, Calibration, and Distractor Effects in Large Language Models](https://mlsanity.com/paper/2502.11028/) (2025)
- [ConfTuner: Training Large Language Models to Express Their Confidence Verbally](https://mlsanity.com/paper/2508.18847/) (2025)
- [TruthRL: Incentivizing Truthful LLMs via Reinforcement Learning](https://mlsanity.com/paper/2509.25760/) (2025)
- [How do LLMs Compute Verbal Confidence](https://mlsanity.com/paper/2603.17839/) (2026)
- [Mitigating LLM Hallucination via Behaviorally Calibrated Reinforcement Learning](https://mlsanity.com/paper/2512.19920/) (2025)
- [Refusal Tokens: A Simple Way to Calibrate Refusals in Large Language Models](https://mlsanity.com/paper/2412.06748/) (2024)
- [Detecting (Un)answerability in Large Language Models with Linear Directions](https://mlsanity.com/paper/2509.22449/) (2025)
- [Fine-Tuning Large Language Models to Appropriately Abstain with Semantic Entropy](https://mlsanity.com/paper/2410.17234/) (2024)
- [ADVICE: Answer-Dependent Verbalized Confidence Estimation](https://mlsanity.com/paper/2510.10913/) (2025)
- [When Robots Should Say "I Don't Know": Benchmarking Abstention in Embodied Question Answering](https://mlsanity.com/paper/2512.04597/) (2025)
- [SAFER: Risk-Constrained Sample-then-Filter in Large Language Models](https://mlsanity.com/paper/2510.10193/) (2025)
- [Are LLM Decisions Faithful to Verbal Confidence?](https://mlsanity.com/paper/2601.07767/) (2026)
- [Answer, Refuse, or Guess? Investigating Risk-Aware Decision Making in Language Models](https://mlsanity.com/paper/2503.01332/) (2025)

## Built on by (most cited first, 5 of 5)

- [Learning When to Remember: Risk-Sensitive Contextual Bandits for Abstention-Aware Memory Retrieval in LLM-Based Coding Agents](https://mlsanity.com/paper/2604.27283/) (2026)
- [Beyond Confidence: Rethinking Self-Assessments for Performance Prediction in LLMs](https://mlsanity.com/paper/2605.07806/) (2026)
- [Don't Start What You Can't Finish: A Counterfactual Audit of Support-State Triage in LLM Agents](https://mlsanity.com/paper/2604.16752/) (2026)
- [Second Guess: Detecting Uncertainty Through Abstention and Answer Stability in Small Language Models](https://mlsanity.com/paper/2605.25394/) (2026)
- [Can Language Models Learn When to Abstain? Policy Learning and Calibrated Answer Selection](https://mlsanity.com/paper/2610.13821/) (2026)

## Abstract (arXiv)

Large language models (LLMs) often produce confident but incorrect answers, in part because standard evaluation incentives reward guessing over expressing uncertainty. We study epistemic abstention for factual questions with verifiable answers, where the goal is to improve selective answering, making LLMs abstain when they are likely to be wrong while preserving correct answers. Inspired by human behavioral decisions in question answering, we introduce I-CALM, a prompt-level framework for black-box LLMs. I-CALM combines elicited verbal confidence, announced answer/abstain payoffs, and normative guidance emphasizing truthfulness, humility, evidential support, and responsibility. To distinguish targeted abstention from indiscriminate refusal, we use a two-stage evaluation protocol, in which LLMs first choose whether to answer or abstain, and are then forced to provide a best guess for the abstained ones. Across models and factual QA datasets, I-CALM improves selective answering by reducing false-answer rate among surfaced responses and improving abstention quality, shifting error-prone cases into abstention while retaining answers the model would have answered correctly. Overall, I-CALM offers a lightweight way to improve inference-time selective answering without retraining or access to model's internal states. Code is available at https://github.com/FayLONG03/hallucinationControl.