mlsanityData to 14 Oct 2026

InternalInspector I2: Robust Confidence Estimation in LLMs through Internal States

arXiv 2406.12053 ↗First posted 17 Jun 2024cs.CLcited by 30

Not on the map. Why?

Abstract (arXiv)

Despite their vast capabilities, Large Language Models (LLMs) often struggle with generating reliable outputs, frequently producing high-confidence inaccuracies known as hallucinations. Addressing this challenge, our research introduces InternalInspector, a novel framework designed to enhance confidence estimation in LLMs by leveraging contrastive learning on internal states including attention states, feed-forward states, and activation states of all layers. Unlike existing methods that primarily focus on the final activation state, InternalInspector conducts a comprehensive analysis across all internal states of every layer to accurately identify both correct and incorrect prediction processes. By benchmarking InternalInspector against existing confidence estimation methods across various natural language understanding and generation tasks, including factual question answering, commonsense reasoning, and reading comprehension, InternalInspector achieves significantly higher accuracy in aligning the estimated confidence scores with the correctness of the LLM's predictions and lower calibration error. Furthermore, InternalInspector excels at HaluEval, a hallucination detection benchmark, outperforming other internal-based confidence estimation methods in this task.

Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). Builds on 48 of its 69 references that are in the map. First posted on arXiv 17 Jun 2024.

Builds on

48 of 69 references.

References · 48

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  6. 2017On Calibration of Modern Neural Networkscited by 4,960
  7. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  8. 2020Supervised Contrastive Learningcited by 2,829
  9. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  10. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  11. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  12. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
36 more
  1. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  2. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  3. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  4. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  5. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,059
  6. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 1,003
  7. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  8. 2019Attention is not Explanationcited by 929
  9. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 856
  10. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 812
  11. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
  12. 2022Teaching Models to Express Their Uncertainty in Wordscited by 751
  13. 2023The Internal State of an LLM Knows When It's Lyingcited by 659
  14. 2019Attention is not not Explanationcited by 636
  15. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 532
  16. 2015An Introduction to Convolutional Neural Networkscited by 514
  17. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 445
  18. 2022Self-critiquing models for assisting human evaluatorscited by 380
  19. 2023A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPTcited by 368
  20. 2020Calibration of Pre-trained Transformerscited by 315
  21. 2020Reducing conversational agents' overconfidence through linguistic calibrationcited by 210
  22. 2022Natural Language to Code Translation with Executioncited by 124
  23. 2023Do Language Models Know When They're Hallucinating References?cited by 116
  24. 2019Learning to Deceive with Attention-Based Explanationscited by 111
  25. 2021Data Augmentation Approaches in Natural Language Processing: A Surveycited by 82
  26. 2019Attention Interpretability Across NLP Taskscited by 71
  27. 2022Measuring the Mixing of Contextual Information in the Transformercited by 65
  28. 2023Adaptation with Self-Evaluation to Improve Selective Prediction in LLMscited by 50
  29. 2022GlobEnc: Quantifying Global Token Attribution by Incorporating the Whole Encoder Layer in Transformerscited by 44
  30. 2023Uncertainty in Natural Language Processing: Sources, Quantification, and Applicationscited by 44
  31. 2023Analyzing Feed-Forward Blocks in Transformers through the Lens of Attention Mapscited by 33
  32. 2023DecompX: Explaining Transformers Decisions by Propagating Token Decompositioncited by 31
  33. 2024Data Augmentation using Large Language Models: Data Perspectives, Learning Paradigms and Challengescited by 30
  34. 2019SANVis: Visual Analytics for Understanding Self-Attention Networkscited by 21
  35. 2019Visualizing Attention in Transformer-Based Language Representation Modelscited by 8
  36. 2023On the Joint Interaction of Models, Data, and Featurescited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 18; 2026: 11.

30 papers.

Sort:

By area:

  1. 2025A Survey on Mechanistic Interpretability for Multi-Modal Foundation Modelscited by 34
  2. 2026Lying with Truths: Open-Channel Multi-Agent Collusion for Belief Manipulation via Generative Montagecited by 19
  3. 2025Too Consistent to Detect: A Study of Self-Consistent Errors in LLMscited by 18
  4. 2025Revisiting Epistemic Markers in Confidence Estimation: Can Markers Accurately Reflect Large Language Models' Uncertainty?cited by 16
  5. 2025Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safetycited by 10
  6. 2025The Geometries of Truth Are Orthogonal Across Taskscited by 9
  7. 2025LLM Knowledge is Brittle: Truthfulness Representations Rely on Superficial Resemblancecited by 9
  8. 2024Rethinking the Uncertainty: A Critical Review and Analysis in the Era of Large Language Modelscited by 6
  9. 2025Self-Evaluating LLMs for Multi-Step Tasks: Stepwise Confidence Estimation for Failure Detectioncited by 6
  10. 2025REMA: A Unified Reasoning Manifold Framework for Interpreting Large Language Modelcited by 5
  11. 2025Generalized Correctness Models: Learning Calibrated and Model-Agnostic Correctness Predictors from Historical Patternscited by 5
  12. 2025On LLMs' Internal Representation of Code Correctnesscited by 5
18 more
  1. 2026Gradient-Stable Attention Heads Signal LLM Correctnesscited by 5
  2. 2025Lightweight Latent Verifiers for Efficient Meta-Generation Strategiescited by 3
  3. 2025What are Models Thinking about? Understanding Large Language Model Hallucinations "Psychology" through Model Inner State Analysiscited by 2
  4. 2025Bi-directional Model Cascading with Proxy Confidencecited by 2
  5. 2025From Hallucinations to Jailbreaks: Rethinking the Vulnerability of Large Foundation Modelscited by 2
  6. 2025Interpreting and Mitigating Unwanted Uncertainty in LLMscited by 2
  7. 2026Calibrated Triage, Not Autonomy: Confidence Estimation for Medical Vision-Language Modelscited by 2
  8. 2026Beyond In-Domain Detection: SpikeScore for Cross-Domain Hallucination Detectioncited by 1
  9. 2026Reading Calibrated Uncertainty from Language Model Trajectoriescited by 1
  10. 2026The ACUTE Protocol: Operationalizing Language Model Activations for Better Calibration, Utility, and Trustcited by 1
  11. 2025Toward Better Generalisation in Uncertainty Estimators: Leveraging Data-Agnostic Featurescited by 0
  12. 2025Enhancing LLM Reasoning via Non-Human-Like Reasoning Path Preference Optimizationcited by 0
  13. 2025Hallucination Detection via Internal States and Structured Reasoning Consistency in Large Language Modelscited by 0
  14. 2026Harnessing the Latent Space: From Steering Vectors to Model Calibrators for Control and Trustcited by 0
  15. 2026Code-MUE: Measuring Code LLMs' Uncertainty through Execution-based Semantic Interaction Graphscited by 0
  16. 2026Vision-Language Model Confidence Is Not a Property of the Answercited by 0
  17. 2026HalluTracer: Pre-Decoding Truthfulness Prediction via Depth-Averaged Probe-Logitcited by 0
  18. 2026VERA: Verdict-Conditioned Reliability for Adaptive LLM Judgescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Enhancing Uncertainty Estimation in LLMs with Expectation of Aggregated Internal Belief
  2. 2026When Words Fall Short: Iterative Synergy Between Verbalized Reasoning and Hidden Features for LLM Confidence Estimation
  3. 2026Confident in a Confidence Score: Investigating the Sensitivity of Confidence Scores to Supervised Fine-Tuning
  4. 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations
  5. 2024Confidence Under the Hood: An Investigation into the Confidence-Probability Alignment in Large Language Models
  6. 2026ConfidenceBench: Evaluating Confidence Calibration in Large Language Models
  7. 2025LENS: Learning Ensemble Confidence from Neural States for Multi-LLM Answer Integration
  8. 2025Taming Object Hallucinations with Verified Atomic Confidence Estimation