mlsanityData to 14 Oct 2026

Towards Fully Exploiting LLM Internal States to Enhance Knowledge Boundary Perception

arXiv 2502.11677 ↗First posted 17 Feb 2025cs.CLcited by 27

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) exhibit impressive performance across diverse tasks but often struggle to accurately gauge their knowledge boundaries, leading to confident yet incorrect responses. This paper explores leveraging LLMs' internal states to enhance their perception of knowledge boundaries from efficiency and risk perspectives. We investigate whether LLMs can estimate their confidence using internal states before response generation, potentially saving computational resources. Our experiments on datasets like Natural Questions, HotpotQA, and MMLU reveal that LLMs demonstrate significant pre-generation perception, which is further refined post-generation, with perception gaps remaining stable across varying conditions. To mitigate risks in critical domains, we introduce Confidence Consistency-based Calibration (C3), which assesses confidence consistency through question reformulation. C3 significantly improves LLMs' ability to recognize their knowledge gaps, enhancing the unknown perception rate by 5.6% on NQ and 4.9% on HotpotQA. Our findings suggest that pre-generation confidence estimation can optimize efficiency, while C3 effectively controls output risks, advancing the reliability of LLMs in practical applications.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Builds on 17 of its 26 references that are in the map. First posted on arXiv 17 Feb 2025.

Builds on

17 of 26 references.

References · 17

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2017On Calibration of Modern Neural Networkscited by 4,960
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  7. 2024Qwen2 Technical Reportcited by 2,120
  8. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  9. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  10. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 812
  11. 2022Teaching Models to Express Their Uncertainty in Wordscited by 751
  12. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 445
5 more
  1. 2020Calibration of Pre-trained Transformerscited by 315
  2. 2024To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoningcited by 263
  3. 2022Prompting GPT-3 To Be Reliablecited by 260
  4. 2024When Do LLMs Need Retrieval Augmentation? Mitigating LLMs' Overconfidence Helps Retrieval Augmentationcited by 59
  5. 2024Are Large Language Models More Honest in Their Probabilistic or Verbalized Confidence?cited by 23

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 14; 2026: 13.

27 papers.

Sort:

By area:

  1. 2025Parameters vs. Context: Fine-Grained Control of Knowledge Reliance in Language Modelscited by 39
  2. 2025RefineX: Learning to Refine Pre-training Data at Scale from Expert-Guided Programscited by 17
  3. 2025Divide-Then-Align: Honest Alignment based on the Knowledge Boundary of RAGcited by 14
  4. 2026Decoupling Reasoning and Confidence: Resurrecting Calibration in Reinforcement Learning from Verifiable Rewardscited by 14
  5. 2025Do LLMs Really Know What They Don't Know? Internal States Mainly Reflect Knowledge Recall Rather Than Truthfulnesscited by 10
  6. 2025KnowGuard: Knowledge-Driven Abstention for Multi-Round Clinical Reasoningcited by 7
  7. 2025Popular but Wrong: Understanding and Mitigating LLM Overconfidence through Knowledge Popularitycited by 6
  8. 2026FACTUM: Mechanistic Detection of Citation Hallucination in Long-Form RAGcited by 6
  9. 2025VeriDispatcher: Multi-Model Dispatching through Pre-Inference Difficulty Prediction for RTL Generation Optimizationcited by 5
  10. 2026Stable-RAG: Mitigating Retrieval-Permutation-Induced Hallucinations in Retrieval-Augmented Generationcited by 5
  11. 2025Do LVLMs Know What They Know? A Systematic Study of Knowledge Boundary Perception in LVLMscited by 4
  12. 2025LLM-Specific Utility for Retrieval-Augmented Generationcited by 4
15 more
  1. 2025Annotation-Efficient Universal Honesty Alignmentcited by 3
  2. 2026Evaluating and Calibrating LLM Confidence on Questions with Multiple Correct Answerscited by 3
  3. 2025Benchmarking Chinese Medical LLMs: A Medbench-based Analysis of Performance Gaps and Hierarchical Optimization Strategiescited by 2
  4. 2025ISACL: Internal State Analyzer for Copyrighted Training Data Leakagecited by 2
  5. 2025Detecting Hallucinations in Graph Retrieval-Augmented Generation via Attention Patterns and Semantic Alignmentcited by 1
  6. 2026GRADE: Probing Knowledge Gaps in LLMs through Gradient Subspace Dynamicscited by 1
  7. 2026Can LLM Rerankers Predict Their Own Ranking Performance?cited by 1
  8. 2025Rethinking LLM Parametric Knowledge as Post-retrieval Confidence for Dynamic Retrieval and Rerankingcited by 0
  9. 2026Probing the Knowledge Boundary: An Interactive Agentic Framework for Deep Knowledge Extractioncited by 0
  10. 2026KA2L: A Knowledge-Aware Active Learning Framework for LLMscited by 0
  11. 2026Logical Consistency as a Bridge: Improving LLM Hallucination Detection via Label Constraint Modeling between Responses and Self-Judgmentscited by 0
  12. 2026The Commit-Abstain Circuit: Why Language Models Hallucinate Instead of Abstainingcited by 0
  13. 2026Calibration, Not Answer Selection: Distilling Internal Confidence in Reasoning Modelscited by 0
  14. 2026Shared Experience, Separate Learning: Companion Confidence Calibration for LLMscited by 0
  15. 2026The Geometry of Knowledge Accessibility in Large Language Modelscited by 0