# Uncertainty Estimation and Quantification for LLMs: A Simple Supervised Approach

arXiv: https://arxiv.org/abs/2404.15993 · HTML: https://mlsanity.com/paper/2404.15993/

## Key facts

- Cited by 76 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 52 references that are in the map.
- First posted on arXiv 24 Apr 2024.

## Builds on (most cited first, 29 of 29)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges](https://mlsanity.com/paper/2011.06225/) (2020)
- [Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback](https://mlsanity.com/paper/2305.14975/) (2023)
- [Discovering Latent Knowledge in Language Models Without Supervision](https://mlsanity.com/paper/2212.03827/) (2022)
- [The Internal State of an LLM Knows When It's Lying](https://mlsanity.com/paper/2304.13734/) (2023)
- [A Survey of Hallucination in Large Foundation Models](https://mlsanity.com/paper/2309.05922/) (2023)
- [INSIDE: LLMs' Internal States Retain the Power of Hallucination Detection](https://mlsanity.com/paper/2402.03744/) (2024)
- [Calibration of Pre-trained Transformers](https://mlsanity.com/paper/2003.07892/) (2020)
- [Unsupervised Quality Estimation for Neural Machine Translation](https://mlsanity.com/paper/2005.10608/) (2020)
- [Reducing conversational agents' overconfidence through linguistic calibration](https://mlsanity.com/paper/2012.14983/) (2020)
- [Conformal Prediction with Large Language Models for Multi-Choice Question Answering](https://mlsanity.com/paper/2305.18404/) (2023)
- [Language Models with Conformal Factuality Guarantees](https://mlsanity.com/paper/2402.10978/) (2024)
- [Decomposing Uncertainty for Large Language Models through Input Clarification Ensembling](https://mlsanity.com/paper/2311.08718/) (2023)
- [Do Androids Know They're Only Dreaming of Electric Sheep?](https://mlsanity.com/paper/2312.17249/) (2023)
- [Cognitive Dissonance: Why Do Language Model Outputs Disagree with Internal Representations of Truthfulness?](https://mlsanity.com/paper/2312.03729/) (2023)
- [Do LLMs Know about Hallucination? An Empirical Investigation of LLM's Hidden States](https://mlsanity.com/paper/2402.09733/) (2024)
- [Distinguishing the Knowable from the Unknowable with Language Models](https://mlsanity.com/paper/2402.03563/) (2024)
- [A Study on the Calibration of In-context Learning](https://mlsanity.com/paper/2312.04021/) (2023)
- [Can Explanations Be Useful for Calibrating Black Box Models?](https://mlsanity.com/paper/2110.07586/) (2021)
- [Probabilities of Chat LLMs Are Miscalibrated but Still Predict Correctness on Multiple-Choice Q&A](https://mlsanity.com/paper/2402.13213/) (2024)
- [Reducing LLM Hallucinations using Epistemic Neural Networks](https://mlsanity.com/paper/2312.15576/) (2023)

## Built on by (most cited first, 30 of 76)

- [A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directions](https://mlsanity.com/paper/2412.05563/) (2024)
- [Uncertainty Quantification and Confidence Calibration in Large Language Models: A Survey](https://mlsanity.com/paper/2503.15850/) (2025)
- [LLM Internal States Reveal Hallucination Risk Faced With a Query](https://mlsanity.com/paper/2407.03282/) (2024)
- [Learning to Route LLMs with Confidence Tokens](https://mlsanity.com/paper/2410.13284/) (2024)
- [A Survey of Uncertainty Estimation Methods on Large Language Models](https://mlsanity.com/paper/2503.00172/) (2025)
- [Latent Space Chain-of-Embedding Enables Output-free LLM Self-Evaluation](https://mlsanity.com/paper/2410.13640/) (2024)
- [Do LLMs estimate uncertainty well in instruction-following?](https://mlsanity.com/paper/2410.14582/) (2024)
- [A Survey on the Honesty of Large Language Models](https://mlsanity.com/paper/2409.18786/) (2024)
- [Revisiting Uncertainty Estimation and Calibration of Large Language Models](https://mlsanity.com/paper/2505.23854/) (2025)
- [From Calibration to Collaboration: LLM Uncertainty Quantification Should Be More Human-Centered](https://mlsanity.com/paper/2506.07461/) (2025)
- [Semantic Volume: Quantifying and Detecting both External and Internal Uncertainty in LLMs](https://mlsanity.com/paper/2502.21239/) (2025)
- [Semantic Energy: Detecting LLM Hallucination Beyond Entropy](https://mlsanity.com/paper/2508.14496/) (2025)
- [A Survey on LLM-based Conversational User Simulation](https://mlsanity.com/paper/2604.24977/) (2026)
- [RadFlag: A Black-Box Hallucination Detection Method for Medical Vision Language Models](https://mlsanity.com/paper/2411.00299/) (2024)
- [Geometric Uncertainty for Detecting and Correcting Hallucinations in LLMs](https://mlsanity.com/paper/2509.13813/) (2025)
- [Fine-Tuning Large Language Models to Appropriately Abstain with Semantic Entropy](https://mlsanity.com/paper/2410.17234/) (2024)
- [Graph-based Confidence Calibration for Large Language Models](https://mlsanity.com/paper/2411.02454/) (2024)
- [Gatekeeper: Improving Model Cascades Through Confidence Tuning](https://mlsanity.com/paper/2502.19335/) (2025)
- [Complementing Self-Consistency with Cross-Model Disagreement for Uncertainty Quantification](https://mlsanity.com/paper/2604.17112/) (2026)
- [LLMs are not Zero-Shot Reasoners for Biomedical Information Extraction](https://mlsanity.com/paper/2408.12249/) (2024)
- [Tools in the Loop: Quantifying Uncertainty of LLM Question Answering Systems That Use Tools](https://mlsanity.com/paper/2505.16113/) (2025)
- [Loki's Dance of Illusions: A Comprehensive Survey of Hallucination in Large Language Models](https://mlsanity.com/paper/2507.02870/) (2025)
- [Transformers Discover Molecular Structure Without Graph Priors](https://mlsanity.com/paper/2510.02259/) (2025)
- [LLM Confidence Evaluation Measures in Zero-Shot CSS Classification](https://mlsanity.com/paper/2410.13047/) (2024)
- [Scaling Textual Gradients via Sampling-Based Momentum](https://mlsanity.com/paper/2506.00400/) (2025)
- [Conformal Prediction Beyond the Seen: A Missing Mass Perspective for Uncertainty Quantification in Generative Models](https://mlsanity.com/paper/2506.05497/) (2025)
- [Uncertainty Quantification for Retrieval-Augmented Reasoning](https://mlsanity.com/paper/2510.11483/) (2025)
- [Confidence Elicitation: A New Attack Vector for Large Language Models](https://mlsanity.com/paper/2502.04643/) (2025)
- [Uncertainty Under the Curve: A Sequence-Level Entropy Area Metric for Reasoning LLM](https://mlsanity.com/paper/2508.20384/) (2025)
- [Generalized Correctness Models: Learning Calibrated and Model-Agnostic Correctness Predictors from Historical Patterns](https://mlsanity.com/paper/2509.24988/) (2025)

## Abstract (arXiv)

In this paper, we study the problem of uncertainty estimation and calibration for LLMs. We begin by formulating the uncertainty estimation problem, a relevant yet underexplored area in existing literature. We then propose a supervised approach that leverages labeled datasets to estimate the uncertainty in LLMs' responses. Based on the formulation, we illustrate the difference between the uncertainty estimation for LLMs and that for standard ML models and explain why the hidden neurons of the LLMs may contain uncertainty information. Our designed approach demonstrates the benefits of utilizing hidden activations to enhance uncertainty estimation across various tasks and shows robust transferability in out-of-distribution settings. We distinguish the uncertainty estimation task from the uncertainty calibration task and show that better uncertainty estimation leads to better calibration performance. Furthermore, our method is easy to implement and adaptable to different levels of model accessibility including black box, grey box, and white box.