# Towards Fully Exploiting LLM Internal States to Enhance Knowledge Boundary Perception

arXiv: https://arxiv.org/abs/2502.11677 · HTML: https://mlsanity.com/paper/2502.11677/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 26 references that are in the map.
- First posted on arXiv 17 Feb 2025.

## Builds on (most cited first, 17 of 17)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs](https://mlsanity.com/paper/2306.13063/) (2023)
- [Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback](https://mlsanity.com/paper/2305.14975/) (2023)
- [Teaching Models to Express Their Uncertainty in Words](https://mlsanity.com/paper/2205.14334/) (2022)
- [How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering](https://mlsanity.com/paper/2012.00955/) (2020)
- [Calibration of Pre-trained Transformers](https://mlsanity.com/paper/2003.07892/) (2020)
- [To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning](https://mlsanity.com/paper/2409.12183/) (2024)
- [Prompting GPT-3 To Be Reliable](https://mlsanity.com/paper/2210.09150/) (2022)
- [When Do LLMs Need Retrieval Augmentation? Mitigating LLMs' Overconfidence Helps Retrieval Augmentation](https://mlsanity.com/paper/2402.11457/) (2024)
- [Are Large Language Models More Honest in Their Probabilistic or Verbalized Confidence?](https://mlsanity.com/paper/2408.09773/) (2024)

## Built on by (most cited first, 27 of 27)

- [Parameters vs. Context: Fine-Grained Control of Knowledge Reliance in Language Models](https://mlsanity.com/paper/2503.15888/) (2025)
- [RefineX: Learning to Refine Pre-training Data at Scale from Expert-Guided Programs](https://mlsanity.com/paper/2507.03253/) (2025)
- [Divide-Then-Align: Honest Alignment based on the Knowledge Boundary of RAG](https://mlsanity.com/paper/2505.20871/) (2025)
- [Decoupling Reasoning and Confidence: Resurrecting Calibration in Reinforcement Learning from Verifiable Rewards](https://mlsanity.com/paper/2603.09117/) (2026)
- [Do LLMs Really Know What They Don't Know? Internal States Mainly Reflect Knowledge Recall Rather Than Truthfulness](https://mlsanity.com/paper/2510.09033/) (2025)
- [KnowGuard: Knowledge-Driven Abstention for Multi-Round Clinical Reasoning](https://mlsanity.com/paper/2509.24816/) (2025)
- [Popular but Wrong: Understanding and Mitigating LLM Overconfidence through Knowledge Popularity](https://mlsanity.com/paper/2505.17537/) (2025)
- [FACTUM: Mechanistic Detection of Citation Hallucination in Long-Form RAG](https://mlsanity.com/paper/2601.05866/) (2026)
- [VeriDispatcher: Multi-Model Dispatching through Pre-Inference Difficulty Prediction for RTL Generation Optimization](https://mlsanity.com/paper/2511.22749/) (2025)
- [Stable-RAG: Mitigating Retrieval-Permutation-Induced Hallucinations in Retrieval-Augmented Generation](https://mlsanity.com/paper/2601.02993/) (2026)
- [Do LVLMs Know What They Know? A Systematic Study of Knowledge Boundary Perception in LVLMs](https://mlsanity.com/paper/2508.19111/) (2025)
- [LLM-Specific Utility for Retrieval-Augmented Generation](https://mlsanity.com/paper/2510.11358/) (2025)
- [Annotation-Efficient Universal Honesty Alignment](https://mlsanity.com/paper/2510.17509/) (2025)
- [Evaluating and Calibrating LLM Confidence on Questions with Multiple Correct Answers](https://mlsanity.com/paper/2602.07842/) (2026)
- [Benchmarking Chinese Medical LLMs: A Medbench-based Analysis of Performance Gaps and Hierarchical Optimization Strategies](https://mlsanity.com/paper/2503.07306/) (2025)
- [ISACL: Internal State Analyzer for Copyrighted Training Data Leakage](https://mlsanity.com/paper/2508.17767/) (2025)
- [Detecting Hallucinations in Graph Retrieval-Augmented Generation via Attention Patterns and Semantic Alignment](https://mlsanity.com/paper/2512.09148/) (2025)
- [GRADE: Probing Knowledge Gaps in LLMs through Gradient Subspace Dynamics](https://mlsanity.com/paper/2604.02830/) (2026)
- [Can LLM Rerankers Predict Their Own Ranking Performance?](https://mlsanity.com/paper/2606.03535/) (2026)
- [Rethinking LLM Parametric Knowledge as Post-retrieval Confidence for Dynamic Retrieval and Reranking](https://mlsanity.com/paper/2509.06472/) (2025)
- [Probing the Knowledge Boundary: An Interactive Agentic Framework for Deep Knowledge Extraction](https://mlsanity.com/paper/2602.00959/) (2026)
- [KA2L: A Knowledge-Aware Active Learning Framework for LLMs](https://mlsanity.com/paper/2603.17566/) (2026)
- [Logical Consistency as a Bridge: Improving LLM Hallucination Detection via Label Constraint Modeling between Responses and Self-Judgments](https://mlsanity.com/paper/2605.03971/) (2026)
- [The Commit-Abstain Circuit: Why Language Models Hallucinate Instead of Abstaining](https://mlsanity.com/paper/2609.32964/) (2026)
- [Calibration, Not Answer Selection: Distilling Internal Confidence in Reasoning Models](https://mlsanity.com/paper/2609.33290/) (2026)
- [Shared Experience, Separate Learning: Companion Confidence Calibration for LLMs](https://mlsanity.com/paper/2609.33721/) (2026)
- [The Geometry of Knowledge Accessibility in Large Language Models](https://mlsanity.com/paper/2610.03052/) (2026)

## Abstract (arXiv)

Large language models (LLMs) exhibit impressive performance across diverse tasks but often struggle to accurately gauge their knowledge boundaries, leading to confident yet incorrect responses. This paper explores leveraging LLMs' internal states to enhance their perception of knowledge boundaries from efficiency and risk perspectives. We investigate whether LLMs can estimate their confidence using internal states before response generation, potentially saving computational resources. Our experiments on datasets like Natural Questions, HotpotQA, and MMLU reveal that LLMs demonstrate significant pre-generation perception, which is further refined post-generation, with perception gaps remaining stable across varying conditions. To mitigate risks in critical domains, we introduce Confidence Consistency-based Calibration (C3), which assesses confidence consistency through question reformulation. C3 significantly improves LLMs' ability to recognize their knowledge gaps, enhancing the unknown perception rate by 5.6% on NQ and 4.9% on HotpotQA. Our findings suggest that pre-generation confidence estimation can optimize efficiency, while C3 effectively controls output risks, advancing the reliability of LLMs in practical applications.