# On Subjective Uncertainty Quantification and Calibration in Natural Language Generation

arXiv: https://arxiv.org/abs/2406.05213 · HTML: https://mlsanity.com/paper/2406.05213/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 58 of its 83 references that are in the map.
- First posted on arXiv 7 Jun 2024.

## Builds on (most cited first, 30 of 58)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs](https://mlsanity.com/paper/2306.13063/) (2023)
- [An Explanation of In-context Learning as Implicit Bayesian Inference](https://mlsanity.com/paper/2111.02080/) (2021)
- [Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation](https://mlsanity.com/paper/2302.09664/) (2023)
- [Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback](https://mlsanity.com/paper/2305.14975/) (2023)
- [Crowdsourcing Multiple Choice Science Questions](https://mlsanity.com/paper/1707.06209/) (2017)
- [Latent Retrieval for Weakly Supervised Open Domain Question Answering](https://mlsanity.com/paper/1906.00300/) (2019)
- [What learning algorithm is in-context learning? Investigations with linear models](https://mlsanity.com/paper/2211.15661/) (2022)
- [CoQA: A Conversational Question Answering Challenge](https://mlsanity.com/paper/1808.07042/) (2018)
- [How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering](https://mlsanity.com/paper/2012.00955/) (2020)
- [Is ChatGPT a Good NLG Evaluator? A Preliminary Study](https://mlsanity.com/paper/2303.04048/) (2023)
- [Generating with Confidence: Uncertainty Quantification for Black-box Large Language Models](https://mlsanity.com/paper/2305.19187/) (2023)
- [Selective Question Answering under Domain Shift](https://mlsanity.com/paper/2006.09462/) (2020)
- [Predict Responsibly: Improving Fairness and Accuracy by Learning to Defer](https://mlsanity.com/paper/1711.06664/) (2017)
- [Reducing conversational agents' overconfidence through linguistic calibration](https://mlsanity.com/paper/2012.14983/) (2020)
- [Many-Shot In-Context Learning](https://mlsanity.com/paper/2404.11018/) (2024)
- [On Hallucination and Predictive Uncertainty in Conditional Language Generation](https://mlsanity.com/paper/2103.15025/) (2021)
- [Conformal Language Modeling](https://mlsanity.com/paper/2306.10193/) (2023)
- [A Theory of Usable Information Under Computational Constraints](https://mlsanity.com/paper/2002.10689/) (2020)
- [Deep Learning for Text Style Transfer: A Survey](https://mlsanity.com/paper/2011.00416/) (2020)
- [Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similarities](https://mlsanity.com/paper/2405.20003/) (2024)

## Built on by (most cited first, 16 of 16)

- [On Verbalized Confidence Scores for LLMs](https://mlsanity.com/paper/2412.14737/) (2024)
- [Uncertainty Quantification for LLMs through Minimum Bayes Risk: Bridging Confidence and Consistency](https://mlsanity.com/paper/2502.04964/) (2025)
- [Conformal Generative Modeling with Improved Sample Efficiency through Sequential Greedy Filtering](https://mlsanity.com/paper/2410.01660/) (2024)
- [Textual Bayes: Quantifying Prompt Uncertainty in LLM-Based Systems](https://mlsanity.com/paper/2506.10060/) (2025)
- [SelfReflect: Can LLMs Communicate Their Internal Answer Distribution?](https://mlsanity.com/paper/2505.20295/) (2025)
- [LLMs Should Express Uncertainty Explicitly](https://mlsanity.com/paper/2604.05306/) (2026)
- [Comparing Uncertainty Measurement and Mitigation Methods for Large Language Models: A Systematic Review](https://mlsanity.com/paper/2504.18346/) (2025)
- [Calibrating Semantic Uncertainty from Observable Language-Model Probabilities](https://mlsanity.com/paper/2607.17447/) (2026)
- [Task-Awareness Improves LLM Generations and Uncertainty](https://mlsanity.com/paper/2601.21500/) (2026)
- [Embedding Trust: Semantic Isotropy Predicts Nonfactuality in Long-Form Text Generation](https://mlsanity.com/paper/2510.21891/) (2025)
- [Conf-Gen: Conformal Uncertainty Quantification for Generative Models](https://mlsanity.com/paper/2605.28920/) (2026)
- [A Formal Framework for Uncertainty Analysis of Text Generation with Large Language Models](https://mlsanity.com/paper/2603.26363/) (2026)
- [Trustworthy deep domain adaptation for wearable photoplethysmography signal analysis with decision-theoretic uncertainty quantification](https://mlsanity.com/paper/2604.17480/) (2026)
- [Concentration and Calibration in Predictive Bayesian Inference](https://mlsanity.com/paper/2605.00455/) (2026)
- [Identification and Honest Recovery from Semantic Observation Kernels: Operator Error, Coarsening, and Stability](https://mlsanity.com/paper/2607.23130/) (2026)
- [Argument Graph Uncertainty: Quantifying Uncertainty from the Logical Structure of Reasoning Chains](https://mlsanity.com/paper/2610.20328/) (2026)

## Abstract (arXiv)

Applications of large language models often involve the generation of free-form responses, in which case uncertainty quantification becomes challenging. This is due to the need to identify task-specific uncertainties (e.g., about the semantics) which appears difficult to define in general cases. This work addresses these challenges from a perspective of Bayesian decision theory, starting from the assumption that our utility is characterized by a similarity measure that compares a generated response with a hypothetical true response. We discuss how this assumption enables principled quantification of the model's subjective uncertainty and its calibration. We further derive a measure for epistemic uncertainty, based on a missing data perspective and its characterization as an excess risk. The proposed methods can be applied to black-box language models. We illustrate the methods on question answering and machine translation tasks. Our experiments provide a principled evaluation of task-specific calibration, and demonstrate that epistemic uncertainty offers a promising deferral strategy for efficient data acquisition in in-context learning.