# Beyond "I Don't Know": Evaluating LLM Self-Awareness in Discriminating Data and Model Uncertainty

arXiv: https://arxiv.org/abs/2604.17293 · HTML: https://mlsanity.com/paper/2604.17293/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 45 references that are in the map.
- First posted on arXiv 19 Apr 2026.

## Builds on (most cited first, 24 of 24)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [HybridFlow: A Flexible and Efficient RLHF Framework](https://mlsanity.com/paper/2409.19256/) (2024)
- [Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning](https://mlsanity.com/paper/2503.09516/) (2025)
- [OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems](https://mlsanity.com/paper/2402.14008/) (2024)
- [GAIA: a benchmark for General AI Assistants](https://mlsanity.com/paper/2311.12983/) (2023)
- [CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing](https://mlsanity.com/paper/2305.11738/) (2023)
- [A Survey of Confidence Estimation and Calibration in Large Language Models](https://mlsanity.com/paper/2311.08298/) (2023)
- [AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions](https://mlsanity.com/paper/2506.09038/) (2025)
- [Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty](https://mlsanity.com/paper/2507.16806/) (2025)
- [SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationales](https://mlsanity.com/paper/2405.20974/) (2024)
- [A Survey on Proactive Dialogue Systems: Problems, Methods, and Prospects](https://mlsanity.com/paper/2305.02750/) (2023)
- [The Curious Case of Hallucinatory (Un)answerability: Finding Truths in the Hidden States of Over-Confident Large Language Models](https://mlsanity.com/paper/2310.11877/) (2023)
- [Distinguishing the Knowable from the Unknowable with Language Models](https://mlsanity.com/paper/2402.03563/) (2024)
- [Benchmarking Hallucination in Large Language Models based on Unanswerable Math Word Problem](https://mlsanity.com/paper/2403.03558/) (2024)
- [Don't Just Say "I don't know"! Self-aligning Large Language Models for Responding to Unknown Questions with Explanations](https://mlsanity.com/paper/2402.15062/) (2024)
- [Adaptive Tool Use in Large Language Models with Meta-Cognition Trigger](https://mlsanity.com/paper/2502.12961/) (2025)
- [WorldSense: A Synthetic Benchmark for Grounded Reasoning in Large Language Models](https://mlsanity.com/paper/2311.15930/) (2023)
- [KnowRL: Exploring Knowledgeable Reinforcement Learning for Factuality](https://mlsanity.com/paper/2506.19807/) (2025)
- [GaRAGe: A Benchmark with Grounding Annotations for RAG Evaluation](https://mlsanity.com/paper/2506.07671/) (2025)
- [Does Biomedical Training Lead to Better Medical Performance?](https://mlsanity.com/paper/2404.04067/) (2024)
- [WiCkeD: A Simple Method to Make Multiple Choice Benchmarks More Challenging](https://mlsanity.com/paper/2502.18316/) (2025)
- [The Dialogue That Heals: A Comprehensive Evaluation of Doctor Agents' Inquiry Capability](https://mlsanity.com/paper/2509.24958/) (2025)
- [Let the Model Distribute Its Doubt: Confidence Estimation through Verbalized Probability Distribution](https://mlsanity.com/paper/2511.14275/) (2025)

## Built on by (most cited first, 2 of 2)

- [Enhancing LLM Metacognition via Cognitive Pairwise Training](https://mlsanity.com/paper/2606.00869/) (2026)
- [Second Guess: Detecting Uncertainty Through Abstention and Answer Stability in Small Language Models](https://mlsanity.com/paper/2605.25394/) (2026)

## Abstract (arXiv)

Reliable Large Language Models (LLMs) should abstain when confidence is insufficient. However, prior studies often treat refusal as a generic "I don't know'', failing to distinguish input-level ambiguity (data uncertainty) from capability limitations (model uncertainty). This lack of distinction limits downstream action decisions like requesting clarification or invoking external tools. In this work, we introduce UA-Bench, a benchmark of over 3,500 questions drawn from six datasets spanning knowledge-intensive and reasoning-intensive tasks, designed to evaluate explicit uncertainty attribution. An evaluation of 18 frontier LLMs shows that even state-of-the-art models struggle to reliably discriminate between data uncertainty and model uncertainty, and that high answer accuracy does not necessarily imply strong uncertainty attribution ability. To narrow this gap, we propose a lightweight data synthesis and reinforcement learning strategy. Experiments on both Qwen3-4B-Instruct-2507 and Qwen3-8B in thinking mode show that the proposed method improves uncertainty attribution while preserving answer accuracy. Our code and data are publicly available now.