mlsanityData to 7 Oct 2026

Estimating LLM Uncertainty with Evidence

arXiv 2502.00290 ↗First posted 1 Feb 2025cs.CL cs.AIcited by 26

Not on the map. Why?

Abstract (arXiv)

Over the past few years, Large Language Models (LLMs) have developed rapidly and are widely applied in various domains. However, LLMs face the issue of hallucinations, generating responses that may be unreliable when the models lack relevant knowledge. To be aware of potential hallucinations, uncertainty estimation methods have been introduced, and most of them have confirmed that reliability lies in critical tokens. However, probability-based methods perform poorly in identifying token reliability, limiting their practical utility. In this paper, we reveal that the probability-based method fails to estimate token reliability due to the loss of evidence strength information which is accumulated in the training stage. Therefore, we present Logits-induced token uncertainty (LogTokU), a framework for estimating decoupled token uncertainty in LLMs, enabling real-time uncertainty estimation without requiring multiple sampling processes. We employ evidence modeling to implement LogTokU and use the estimated uncertainty to guide downstream tasks. The experimental results demonstrate that LogTokU has significant effectiveness and promise.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 25 of its 38 references that are in the map. First posted on arXiv 1 Feb 2025.

Builds on

25 of 38 references.

References · 25

  1. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  2. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  3. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  4. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  5. 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 813
  6. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
  7. 2018Evidential Deep Learning to Quantify Classification Uncertaintycited by 728
  8. 2023Generating with Confidence: Uncertainty Quantification for Black-box Large Language Modelscited by 286
  9. 2022Mitigating Neural Network Overconfidence with Logit Normalizationcited by 251
  10. 2021On Hallucination and Predictive Uncertainty in Conditional Language Generationcited by 176
  11. 2024Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantificationcited by 159
  12. 2023Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Modelscited by 147
13 more
  1. 2024Large Legal Fictions: Profiling Legal Hallucinations in Large Language Modelscited by 144
  2. 2024GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluationscited by 118
  3. 2024Language Model Cascades: Token-level uncertainty and beyondcited by 115
  4. 2024Beyond Functional Correctness: Exploring Hallucinations in LLM-Generated Codecited by 106
  5. 2023Enhancing Uncertainty-Based Hallucination Detection with Stronger Focuscited by 98
  6. 2023SAC3: Reliable Hallucination Detection in Black-Box Language Models via Semantic-aware Cross-check Consistencycited by 89
  7. 2021Types of Out-of-Distribution Texts and How to Detect Themcited by 80
  8. 2024Critical Tokens Matter: Token-Level Contrastive Estimation Enhances LLM's Reasoning Capabilitycited by 80
  9. 2024LLMs Will Always Hallucinate, and We Need to Live With Thiscited by 77
  10. 2021Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimationcited by 71
  11. 2023Selectively Answering Ambiguous Questionscited by 63
  12. 2024A Survey of Uncertainty Estimation in LLMs: Theory Meets Practicecited by 54
  13. 2024Forking Paths in Neural Text Generationcited by 30

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 10; 2026: 16.

26 papers.

Sort:

By area:

  1. 2025Parameters vs. Context: Fine-Grained Control of Knowledge Reliance in Language Modelscited by 38
  2. 2025R2R: Efficiently Navigating Divergent Reasoning Paths with Small-Large Model Token Routingcited by 28
  3. 2025Semantic Energy: Detecting LLM Hallucination Beyond Entropycited by 18
  4. 2025Learned Hallucination Detection in Black-Box LLMs using Token-level Entropy Production Ratecited by 9
  5. 2026Detecting Misbehaviors of Large Vision-Language Models by Evidential Uncertainty Quantificationcited by 5
  6. 2025Can LLMs Detect Their Confabulations? Estimating Reliability in Uncertainty-Aware Language Modelscited by 4
  7. 2026The Alignment Tax: Response Homogenization in Aligned LLMs and Its Implications for Uncertainty Estimationcited by 4
  8. 2026Uncertainty Quantification for Flow-Based Vision-Language-Action Modelscited by 4
  9. 2025INSIGHT: INference-time Sequence Introspection for Generating Help Triggers in Vision-Language-Action Modelscited by 3
  10. 2025ProgRAG: Hallucination-Resistant Progressive Retrieval and Reasoning over Knowledge Graphscited by 3
  11. 2025REX-RAG: Reasoning Exploration with Policy Correction in Retrieval-Augmented Generationcited by 2
  12. 2026WebUncertainty: Dual-Level Uncertainty Driven Planning and Reasoning For Autonomous Web Agentcited by 2
14 more
  1. 2026Towards Generation-Efficient Uncertainty Estimation in Large Language Modelscited by 2
  2. 2026Quantifying Consistency in LLM Logical Reasoning via Structural Uncertaintycited by 2
  3. 2025DND: Boosting Large Language Models with Dynamic Nested Depthcited by 1
  4. 2025Calibrating Uncertainty for Zero-Shot Adversarial CLIPcited by 1
  5. 2026Beyond Confidence: The Rhythms of Reasoning in Generative Modelscited by 0
  6. 2026Uncertainty-Aware Exploratory Direct Preference Optimization for Multimodal Large Language Modelscited by 0
  7. 2026VIHD: Visual Intervention-based Hallucination Detection for Medical Visual Question Answeringcited by 0
  8. 2026Integrating Local and Global Entropy for Uncertainty Quantification in LLMscited by 0
  9. 2026Adaptive Nucleus Truncation for Long-Form Reasoningcited by 0
  10. 2026LLM Doesn't Know What It Doesn't Know: Detecting Epistemic Blind Spots via Cross-Model Attribution Divergence on Clinical Tabular Datacited by 0
  11. 2026TrustVLA: Mechanism-Guided Inference-Time Defense Against Vision-Language-Action Backdoorscited by 0
  12. 2026ProbPlug: A Plugin Uncertainty Network for Reliable Confidence in LLM Binary Classificationcited by 0
  13. 2026Rethinking Correctness for Uncertainty Estimation in Clinical Prediction with Vision-Language Modelscited by 0
  14. 2026Generalization Is Stability, Not Accuracy: Multi-Axis Evaluation of LLMscited by 0