# A Formal Framework for Uncertainty Analysis of Text Generation with Large Language Models

arXiv: https://arxiv.org/abs/2603.26363 · HTML: https://mlsanity.com/paper/2603.26363/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 30 references that are in the map.
- First posted on arXiv 27 Mar 2026.

## Builds on (most cited first, 27 of 27)

- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation](https://mlsanity.com/paper/2302.09664/) (2023)
- [SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models](https://mlsanity.com/paper/2303.08896/) (2023)
- [Teaching Models to Express Their Uncertainty in Words](https://mlsanity.com/paper/2205.14334/) (2022)
- [How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering](https://mlsanity.com/paper/2012.00955/) (2020)
- [Hallucination is Inevitable: An Innate Limitation of Large Language Models](https://mlsanity.com/paper/2401.11817/) (2024)
- [Why Language Models Hallucinate](https://mlsanity.com/paper/2509.04664/) (2025)
- [Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantification](https://mlsanity.com/paper/2403.04696/) (2024)
- [Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similarities](https://mlsanity.com/paper/2405.20003/) (2024)
- [Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Models](https://mlsanity.com/paper/2307.01379/) (2023)
- [Calibrated Language Models Must Hallucinate](https://mlsanity.com/paper/2311.14648/) (2023)
- [Enhancing Uncertainty-Based Hallucination Detection with Stronger Focus](https://mlsanity.com/paper/2311.13230/) (2023)
- [Uncertainty Quantification and Confidence Calibration in Large Language Models: A Survey](https://mlsanity.com/paper/2503.15850/) (2025)
- [A Survey of Uncertainty Estimation Methods on Large Language Models](https://mlsanity.com/paper/2503.00172/) (2025)
- [Uncertainty Quantification for In-Context Learning of Large Language Models](https://mlsanity.com/paper/2402.10189/) (2024)
- [Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Models](https://mlsanity.com/paper/2307.10236/) (2023)
- [Uncertainty quantification in fine-tuned LLMs using LoRA ensembles](https://mlsanity.com/paper/2402.12264/) (2024)
- [Understanding the Uncertainty of LLM Explanations: A Perspective Based on Reasoning Topology](https://mlsanity.com/paper/2502.17026/) (2025)
- [Black-box Uncertainty Quantification Method for LLM-as-a-Judge](https://mlsanity.com/paper/2410.11594/) (2024)
- [On the Effect of Sampling Diversity in Scaling LLM Inference](https://mlsanity.com/paper/2502.11027/) (2025)
- [On Subjective Uncertainty Quantification and Calibration in Natural Language Generation](https://mlsanity.com/paper/2406.05213/) (2024)
- [A Human-AI Comparative Analysis of Prompt Sensitivity in LLM-Based Relevance Judgment](https://mlsanity.com/paper/2504.12408/) (2025)
- [Randomness Is All You Need: Semantic Traversal of Problem-Solution Spaces with Large Language Models](https://mlsanity.com/paper/2402.06053/) (2024)

## Abstract (arXiv)

The generation of texts using Large Language Models (LLMs) is inherently uncertain, with sources of uncertainty being not only the generation of texts, but also the prompt used and the downstream interpretation. Within this work, we provide a formal framework for the measurement of uncertainty that takes these different aspects into account. Our framework models prompting, generation, and interpretation as interconnected autoregressive processes that can be combined into a single sampling tree. We introduce filters and objective functions to describe how different aspects of uncertainty can be expressed over the sampling tree and demonstrate how to express existing approaches towards uncertainty through these functions. With our framework we show not only how different methods are formally related and can be reduced to a common core, but also point out additional aspects of uncertainty that have not yet been studied.