# Examining LLMs' Uncertainty Expression Towards Questions Outside Parametric Knowledge

arXiv: https://arxiv.org/abs/2311.09731 · HTML: https://mlsanity.com/paper/2311.09731/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 43 references that are in the map.
- First posted on arXiv 16 Nov 2023.

## Builds on (most cited first, 28 of 28)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs](https://mlsanity.com/paper/2306.13063/) (2023)
- [Teaching Models to Express Their Uncertainty in Words](https://mlsanity.com/paper/2205.14334/) (2022)
- [Learning the Difference that Makes a Difference with Counterfactually-Augmented Data](https://mlsanity.com/paper/1909.12434/) (2019)
- [A Survey of Hallucination in Large Foundation Models](https://mlsanity.com/paper/2309.05922/) (2023)
- [FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation](https://mlsanity.com/paper/2310.03214/) (2023)
- [Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks](https://mlsanity.com/paper/2307.02477/) (2023)
- [Reducing conversational agents' overconfidence through linguistic calibration](https://mlsanity.com/paper/2012.14983/) (2020)
- [A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generation](https://mlsanity.com/paper/2307.03987/) (2023)
- [Language models show human-like content effects on reasoning tasks](https://mlsanity.com/paper/2207.07051/) (2022)
- [Sources of Hallucination by Large Language Models on Inference Tasks](https://mlsanity.com/paper/2305.14552/) (2023)
- [On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models?](https://mlsanity.com/paper/2204.07931/) (2022)
- [Uncertainty Quantification with Pre-trained Language Models: A Large-Scale Empirical Analysis](https://mlsanity.com/paper/2210.04714/) (2022)
- [Testing the General Deductive Reasoning Capacity of Large Language Models Using OOD Examples](https://mlsanity.com/paper/2305.15269/) (2023)
- [CREPE: Open-Domain Question Answering with False Presuppositions](https://mlsanity.com/paper/2211.17257/) (2022)
- [Uncertainty in Natural Language Generation: From Theory to Applications](https://mlsanity.com/paper/2307.15703/) (2023)
- [From Language Modeling to Instruction Following: Understanding the Behavior Shift in LLMs after Instruction Tuning](https://mlsanity.com/paper/2310.00492/) (2023)
- [Won't Get Fooled Again: Answering Questions with False Premises](https://mlsanity.com/paper/2307.02394/) (2023)
- [Improving the Reliability of Large Language Models by Leveraging Uncertainty-Aware In-Context Learning](https://mlsanity.com/paper/2310.04782/) (2023)
- [Counterfactual reasoning: Testing language models' understanding of hypothetical scenarios](https://mlsanity.com/paper/2305.16572/) (2023)
- [Augmenting LLMs with Knowledge: A survey on hallucination prevention](https://mlsanity.com/paper/2309.16459/) (2023)

## Built on by (most cited first, 30 of 32)

- [X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agents](https://mlsanity.com/paper/2504.13203/) (2025)
- [Knowledge Boundary of Large Language Models: A Survey](https://mlsanity.com/paper/2412.12472/) (2024)
- [Generalization Bias in Large Language Model Summarization of Scientific Research](https://mlsanity.com/paper/2504.00025/) (2025)
- [MOSAIC: Modeling Social AI for Content Dissemination and Regulation in Multi-Agent Simulations](https://mlsanity.com/paper/2504.07830/) (2025)
- [BeHonest: Benchmarking Honesty in Large Language Models](https://mlsanity.com/paper/2406.13261/) (2024)
- [A Survey on the Honesty of Large Language Models](https://mlsanity.com/paper/2409.18786/) (2024)
- [Step-On-Feet Tuning: Scaling Self-Alignment of LLMs via Bootstrapping](https://mlsanity.com/paper/2402.07610/) (2024)
- [UAlign: Leveraging Uncertainty Estimations for Factuality Alignment on Large Language Models](https://mlsanity.com/paper/2412.11803/) (2024)
- [LLMs' Reading Comprehension Is Affected by Parametric Knowledge and Struggles with Hypothetical Statements](https://mlsanity.com/paper/2404.06283/) (2024)
- [Cannot or Should Not? Automatic Analysis of Refusal Composition in IFT/RLHF Datasets and Refusal Behavior of Black-Box LLMs](https://mlsanity.com/paper/2412.16974/) (2024)
- [MMBoundary: Advancing MLLM Knowledge Boundary Awareness through Reasoning Step Confidence Calibration](https://mlsanity.com/paper/2505.23224/) (2025)
- [MoHoBench: Assessing Honesty of Multimodal Large Language Models via Unanswerable Visual Questions](https://mlsanity.com/paper/2507.21503/) (2025)
- [Uncertainty Quantification in Retrieval Augmented Question Answering](https://mlsanity.com/paper/2502.18108/) (2025)
- [SafeTy Reasoning Elicitation Alignment for Multi-Turn Dialogues](https://mlsanity.com/paper/2506.00668/) (2025)
- [LLMs can learn self-restraint through iterative self-reflection](https://mlsanity.com/paper/2405.13022/) (2024)
- [A Unified Definition of Hallucination: It's The World Model, Stupid!](https://mlsanity.com/paper/2512.21577/) (2025)
- [Recognizing Limits: Investigating Infeasibility in Large Language Models](https://mlsanity.com/paper/2408.05873/) (2024)
- [Confidence Calibration for Multimodal LLMs: An Empirical Study through Medical VQA](https://mlsanity.com/paper/2606.19950/) (2026)
- [Fine-Tuned LLMs Know They Don't Know: A Parameter-Efficient Approach to Recovering Honesty](https://mlsanity.com/paper/2511.12991/) (2025)
- [SciPredict: Can LLMs Predict the Outcomes of Scientific Experiments in Natural Sciences?](https://mlsanity.com/paper/2604.10718/) (2026)
- [Probing LLM Hallucination from Within: Perturbation-Driven Approach via Internal Knowledge](https://mlsanity.com/paper/2411.09689/) (2024)
- [Knowledge Graph Guided Evaluation of Abstention Techniques](https://mlsanity.com/paper/2412.07430/) (2024)
- [Prioritizing Image-Related Tokens Enhances Vision-Language Pre-Training](https://mlsanity.com/paper/2505.08971/) (2025)
- [PhantomBench: Benchmarking the Non-existential Threat of Language Models](https://mlsanity.com/paper/2606.11105/) (2026)
- [Knowledge Boundary Discovery for Large Language Models](https://mlsanity.com/paper/2603.21022/) (2026)
- [Learning from AVA: Early Lessons from a Curated and Trustworthy Generative AI for Policy and Development Research](https://mlsanity.com/paper/2604.17843/) (2026)
- [Unlearners Can Lie: Evaluating and Improving Honesty in LLM Unlearning](https://mlsanity.com/paper/2605.08765/) (2026)
- [PCS: Perceived Confidence Scoring of Black Box LLMs with Metamorphic Relations](https://mlsanity.com/paper/2502.07186/) (2025)
- [What Models Know, How Well They Know It: Knowledge-Weighted Fine-Tuning for Learning When to Say "I Don't Know"](https://mlsanity.com/paper/2604.05779/) (2026)
- [Automatic Hard Example Synthesis with Multi-Level Agentic Data Curation](https://mlsanity.com/paper/2607.14256/) (2026)

## Abstract (arXiv)

Can large language models (LLMs) express their uncertainty in situations where they lack sufficient parametric knowledge to generate reasonable responses? This work aims to systematically investigate LLMs' behaviors in such situations, emphasizing the trade-off between honesty and helpfulness. To tackle the challenge of precisely determining LLMs' knowledge gaps, we diagnostically create unanswerable questions containing non-existent concepts or false premises, ensuring that they are outside the LLMs' vast training data. By compiling a benchmark, UnknownBench, which consists of both unanswerable and answerable questions, we quantitatively evaluate the LLMs' performance in maintaining honesty while being helpful. Using a model-agnostic unified confidence elicitation approach, we observe that most LLMs fail to consistently refuse or express uncertainty towards questions outside their parametric knowledge, although instruction fine-tuning and alignment techniques can provide marginal enhancements. Moreover, LLMs' uncertainty expression does not always stay consistent with the perceived confidence of their textual outputs.