mlsanityData to 7 Oct 2026

A Formal Framework for Uncertainty Analysis of Text Generation with Large Language Models

arXiv 2603.26363 ↗First posted 27 Mar 2026cs.LG cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

The generation of texts using Large Language Models (LLMs) is inherently uncertain, with sources of uncertainty being not only the generation of texts, but also the prompt used and the downstream interpretation. Within this work, we provide a formal framework for the measurement of uncertainty that takes these different aspects into account. Our framework models prompting, generation, and interpretation as interconnected autoregressive processes that can be combined into a single sampling tree. We introduce filters and objective functions to describe how different aspects of uncertainty can be expressed over the sampling tree and demonstrate how to express existing approaches towards uncertainty through these functions. With our framework we show not only how different methods are formally related and can be reduced to a common core, but also point out additional aspects of uncertainty that have not yet been studied.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 27 of its 30 references that are in the map. First posted on arXiv 27 Mar 2026.

Builds on

27 of 30 references.

References · 27

  1. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  2. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  3. 2024A Survey on LLM-as-a-Judgecited by 1,384
  4. 2018Hierarchical Neural Story Generationcited by 1,182
  5. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 903
  6. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
  7. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  8. 2022Teaching Models to Express Their Uncertainty in Wordscited by 731
  9. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 439
  10. 2024Hallucination is Inevitable: An Innate Limitation of Large Language Modelscited by 329
  11. 2025Why Language Models Hallucinatecited by 205
  12. 2024Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantificationcited by 159
15 more
  1. 2024Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similaritiescited by 156
  2. 2023Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Modelscited by 147
  3. 2023Calibrated Language Models Must Hallucinatecited by 114
  4. 2023Enhancing Uncertainty-Based Hallucination Detection with Stronger Focuscited by 98
  5. 2025Uncertainty Quantification and Confidence Calibration in Large Language Models: A Surveycited by 93
  6. 2025A Survey of Uncertainty Estimation Methods on Large Language Modelscited by 57
  7. 2024Uncertainty Quantification for In-Context Learning of Large Language Modelscited by 45
  8. 2023Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Modelscited by 40
  9. 2024Uncertainty quantification in fine-tuned LLMs using LoRA ensemblescited by 35
  10. 2025Understanding the Uncertainty of LLM Explanations: A Perspective Based on Reasoning Topologycited by 24
  11. 2024Black-box Uncertainty Quantification Method for LLM-as-a-Judgecited by 18
  12. 2025On the Effect of Sampling Diversity in Scaling LLM Inferencecited by 18
  13. 2025A Human-AI Comparative Analysis of Prompt Sensitivity in LLM-Based Relevance Judgmentcited by 15
  14. 2024On Subjective Uncertainty Quantification and Calibration in Natural Language Generationcited by 14
  15. 2024Randomness Is All You Need: Semantic Traversal of Problem-Solution Spaces with Large Language Modelscited by 3

Built on by

Not cited yet.