mlsanityData to 14 Oct 2026

ESI: Epistemic Uncertainty Quantification via Semantic-preserving Intervention for Large Language Models

arXiv 2510.13103 ↗First posted 15 Oct 2025cs.CL cs.AI cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Uncertainty Quantification (UQ) is a promising approach to improve model reliability, yet quantifying the uncertainty of Large Language Models (LLMs) is non-trivial. In this work, we establish a connection between the uncertainty of LLMs and their invariance under semantic-preserving intervention from a causal perspective. Building on this foundation, we propose a novel grey-box uncertainty quantification method that measures the variation in model outputs before and after the semantic-preserving intervention. Through theoretical justification, we show that our method provides an effective estimate of epistemic uncertainty. Our extensive experiments, conducted across various LLMs and a variety of question-answering (QA) datasets, demonstrate that our method excels not only in terms of effectiveness but also in computational efficiency.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 33 of its 42 references that are in the map. First posted on arXiv 15 Oct 2025.

Builds on

33 of 42 references.

References · 33

  1. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  2. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  3. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  4. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  5. 2020Shortcut Learning in Deep Neural Networkscited by 1,969
  6. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  7. 2019Invariant Risk Minimizationcited by 1,854
  8. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  9. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
  10. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 913
  11. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 819
  12. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 812
21 more
  1. 2017Crowdsourcing Multiple Choice Science Questionscited by 765
  2. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
  3. 2018CoQA: A Conversational Question Answering Challengecited by 683
  4. 2021A Survey of Uncertainty in Deep Neural Networkscited by 680
  5. 2023A Comprehensive Overview of Large Language Modelscited by 648
  6. 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 390
  7. 2024INSIDE: LLMs' Internal States Retain the Power of Hallucination Detectioncited by 323
  8. 2023Generating with Confidence: Uncertainty Quantification for Black-box Large Language Modelscited by 293
  9. 2020Reducing conversational agents' overconfidence through linguistic calibrationcited by 210
  10. 2021On Hallucination and Predictive Uncertainty in Conditional Language Generationcited by 176
  11. 2023Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Modelscited by 150
  12. 2024Language Models with Conformal Factuality Guaranteescited by 131
  13. 2024A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directionscited by 121
  14. 2024To Believe or Not to Believe Your LLMcited by 120
  15. 2023Decomposing Uncertainty for Large Language Models through Input Clarification Ensemblingcited by 114
  16. 2022Locally Typical Samplingcited by 111
  17. 2023Evaluating Open-Domain Question Answering in the Era of Large Language Modelscited by 111
  18. 2022Tomayto, Tomahto. Beyond Token-level Answer Equivalence for Question Answering Evaluationcited by 67
  19. 2023Uncertainty in Natural Language Generation: From Theory to Applicationscited by 61
  20. 2023Introducing an Improved Information-Theoretic Measure of Predictive Uncertaintycited by 28
  21. 2024Causal-Guided Active Learning for Debiasing Large Language Modelscited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Credal Large Language Models for Semantic Commitment under Uncertaintycited by 2
  2. 2026XU-RS: Explaining Credal Width in Random-Set Language Modelscited by 0