mlsanityData to 14 Oct 2026

Factual Confidence of LLMs: on Reliability and Robustness of Current Estimators

arXiv 2406.13415 ↗First posted 19 Jun 2024cs.CL cs.LGcited by 61

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) tend to be unreliable in the factuality of their answers. To address this problem, NLP researchers have proposed a range of techniques to estimate LLM's confidence over facts. However, due to the lack of a systematic comparison, it is not clear how the different methods compare to one another. To fill this gap, we present a survey and empirical comparison of estimators of factual confidence. We define an experimental framework allowing for fair comparison, covering both fact-verification and question answering. Our experiments across a series of LLMs indicate that trained hidden-state probes provide the most reliable confidence estimates, albeit at the expense of requiring access to weights and training data. We also conduct a deeper assessment of factual confidence by measuring the consistency of model behavior under meaning-preserving variations in the input. We find that the confidence of LLMs is often unstable across semantically equivalent inputs, suggesting that there is much room for improvement of the stability of models' parametric knowledge. Our code is available at (https://github.com/amazon-science/factual-confidence-of-llms).

Key facts. Cited by 61 papers in the map (data through 14 Oct 2026). Builds on 35 of its 48 references that are in the map. First posted on arXiv 19 Jun 2024.

Builds on

35 of 48 references.

References · 35

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  4. 2017On Calibration of Modern Neural Networkscited by 4,960
  5. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  6. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  7. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  8. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  9. 2019Language Models as Knowledge Bases?cited by 1,936
  10. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  11. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  12. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
23 more
  1. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  2. 2021Ethical and social risks of harm from Language Modelscited by 966
  3. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 812
  4. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
  5. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  6. 2022Teaching Models to Express Their Uncertainty in Wordscited by 751
  7. 2023The Falcon Series of Open Language Modelscited by 535
  8. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 445
  9. 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 404
  10. 2021A Survey on Automated Fact-Checkingcited by 378
  11. 2021Measuring and Improving Consistency in Pretrained Language Modelscited by 364
  12. 2023Do Large Language Models Know What They Don't Know?cited by 243
  13. 2020Unsupervised Quality Estimation for Neural Machine Translationcited by 211
  14. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 206
  15. 2021Understanding the Capabilities, Limitations, and Societal Impact of Large Language Modelscited by 160
  16. 2019The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectivescited by 139
  17. 2021Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Modelscited by 132
  18. 2023Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Modelscited by 118
  19. 2021Truthful AI: Developing and governing AI that does not liecited by 113
  20. 2018The Fact Extraction and VERification (FEVER) Shared Taskcited by 108
  21. 2023How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questionscited by 83
  22. 2021A Survey on Stance Detection for Mis- and Disinformation Identificationcited by 75
  23. 2023Separating form and meaning: Using self-consistency to quantify task understanding across multiple sensescited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 7; 2025: 27; 2026: 27.

61 papers.

Sort:

By area:

  1. 2024A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directionscited by 121
  2. 2024Learning to Route LLMs with Confidence Tokenscited by 67
  3. 2024Probing-RAG: Self-Probing to Guide Language Models in Selective Document Retrievalcited by 29
  4. 2024Atomic Calibration of LLMs in Long-Form Generationscited by 24
  5. 2024A Survey on the Honesty of Large Language Modelscited by 23
  6. 2025Chain-of-Thought Prompting Obscures Hallucination Cues in Large Language Models: An Empirical Evaluationcited by 21
  7. 2026Dynamic Model Routing and Cascading for Efficient LLM Inference: A Surveycited by 19
  8. 2025Too Consistent to Detect: A Study of Self-Consistent Errors in LLMscited by 18
  9. 2025Probing the Geometry of Truth: Consistency and Generalization of Truth Directions in LLMs Across Logical Transformations and Question Answering Taskscited by 18
  10. 2026Auditing Multi-Agent LLM Reasoning Trees Outperforms Majority Vote and LLM-as-Judgecited by 18
  11. 2024VideoICL: Confidence-based Iterative In-context Learning for Out-of-Distribution Video Understandingcited by 17
  12. 2025Reconsidering LLM Uncertainty Estimation Methods in the Wildcited by 17
38 more
  1. 2025OThink-R1: Intrinsic Fast/Slow Thinking Mode Switching for Over-Reasoning Mitigationcited by 17
  2. 2025Rethinking the Reliability of Multi-agent System: A Perspective from Byzantine Fault Tolerancecited by 17
  3. 2025Control the Temperature: Selective Sampling for Diverse and High-Quality LLM Outputscited by 15
  4. 2025Query-Level Uncertainty in Large Language Modelscited by 14
  5. 2025SynAdapt: Learning Adaptive Reasoning in Large Language Models via Synthetic Continuous Chain-of-Thoughtcited by 13
  6. 2025Robustness in Large Language Models: A Survey of Mitigation Strategies and Evaluation Metricscited by 12
  7. 2025Cascaded Language Models for Cost-effective Human-AI Decision-Makingcited by 11
  8. 2025Gatekeeper: Improving Model Cascades Through Confidence Tuningcited by 10
  9. 2025Failure Modes in LLM Systems: A System-Level Taxonomy for Reliable AI Applicationscited by 10
  10. 2026BaseCal: Unsupervised Confidence Calibration via Base Model Signalscited by 8
  11. 2025Teaching Language Models to Faithfully Express their Uncertaintycited by 7
  12. 2025Bridging the Knowledge-Prediction Gap in LLMs on Multiple-Choice Questionscited by 6
  13. 2024On the attribution of confidence to large language modelscited by 5
  14. 2025CheXPO: Preference Optimization for Chest X-ray VLMs with Counterfactual Rationalecited by 5
  15. 2025Co-Investigator AI: The Rise of Agentic AI for Smarter, Trustworthy AML Compliance Narrativescited by 4
  16. 2025Bridging External and Parametric Knowledge: Mitigating Hallucination of LLMs with Shared-Private Semantic Synergy in Dual-Stream Knowledgecited by 3
  17. 2026Semantic Calibration Prevails Where Token Confidence Fails: Benchmarking Long-Form Scientific QAcited by 3
  18. 2026Lyapunov Probes for Hallucination Detection in Large Foundation Modelscited by 3
  19. 2026Probability Contracts: Accuracy, Coherence, and Decisions Across LLM Interfacescited by 3
  20. 2025Maximizing Signal in Human-Model Preference Alignmentcited by 2
  21. 2025When LLMs Team Up: The Emergence of Collaborative Affective Computingcited by 2
  22. 2025From Confidence to Collapse in LLM Factual Robustnesscited by 2
  23. 2025Trustworthy Agents for Electronic Health Records through Confidence Estimationcited by 2
  24. 2025Human or LLM as Standardized Patients? A Comparative Study for Medical Educationcited by 2
  25. 2026From Signals to Transfer: A Factorised Study of Probe-Based Uncertainty Estimation in Large Language Modelscited by 2
  26. 2025Learning Auxiliary Tasks Improves Reference-Free Hallucination Detection in Open-Domain Long-Form Generationcited by 1
  27. 2026Diagnosing LLM Arbitration Behavior over Pre-evidence Epistemic States in RAG-based Fact-Checkingcited by 1
  28. 2026A Calibrated Reflection Approach for Enhancing Confidence Estimation in LLMscited by 1
  29. 2026Memory Control Signals Emerge Before Action in Long Horizon Agentscited by 1
  30. 2025TrustDataFilter:Leveraging Trusted Knowledge Base Data for More Effective Filtering of Unknown Informationcited by 0
  31. 2025Toward Better Generalisation in Uncertainty Estimators: Leveraging Data-Agnostic Featurescited by 0
  32. 2025Intelligence Foundation Model: A New Perspective to Approach Artificial General Intelligencecited by 0
  33. 2026Fact-Checking with Large Language Models via Probabilistic Certainty and Consistencycited by 0
  34. 2026REASON: Accelerating Probabilistic Logical Reasoning for Scalable Neuro-Symbolic Intelligencecited by 0
  35. 2026Nexus: Inferring Join Graphs from Metadata Alone via Iterative Low-Rank Matrix Completioncited by 0
  36. 2026Disentangling Deception and Hallucination Failures in LLMscited by 0
  37. 2026CAST: Achieving Stable LLM-based Text Analysis for Data Analyticscited by 0
  38. 2026Confidence-Driven Multi-Scale Model Selection for Cost-Efficient Inferencecited by 0

50 most cited of 61 All 61 (JSON)