# Mind the Generation Process: Fine-Grained Confidence Estimation During LLM Generation

arXiv: https://arxiv.org/abs/2508.12040 · HTML: https://mlsanity.com/paper/2508.12040/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 43 references that are in the map.
- First posted on arXiv 16 Aug 2025.

## Builds on (most cited first, 28 of 28)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering](https://mlsanity.com/paper/1809.02789/) (2018)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge](https://mlsanity.com/paper/1811.00937/) (2018)
- [Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs](https://mlsanity.com/paper/2306.13063/) (2023)
- [Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback](https://mlsanity.com/paper/2305.14975/) (2023)
- [Teaching Models to Express Their Uncertainty in Words](https://mlsanity.com/paper/2205.14334/) (2022)
- [Leveraging Large Language Models for Multiple Choice Question Answering](https://mlsanity.com/paper/2210.12353/) (2022)
- [The Effect of Sampling Temperature on Problem Solving in Large Language Models](https://mlsanity.com/paper/2402.05201/) (2024)
- [LLM Critics Help Catch LLM Bugs](https://mlsanity.com/paper/2407.00215/) (2024)
- [To Believe or Not to Believe Your LLM](https://mlsanity.com/paper/2406.02543/) (2024)
- [The Good, The Bad, and The Greedy: Evaluation of LLMs Should Not Ignore Non-Determinism](https://mlsanity.com/paper/2407.10457/) (2024)
- [An Empirical Study of LLM-as-a-Judge for LLM Evaluation: Fine-tuned Judge Model is not a General Substitute for GPT-4](https://mlsanity.com/paper/2403.02839/) (2024)
- [Navigating the Grey Area: How Expressions of Uncertainty and Overconfidence Affect Language Models](https://mlsanity.com/paper/2302.13439/) (2023)
- [Can LLMs Learn from Previous Mistakes? Investigating LLMs' Errors to Boost for Reasoning](https://mlsanity.com/paper/2403.20046/) (2024)
- [SelfIE: Self-Interpretation of Large Language Model Embeddings](https://mlsanity.com/paper/2403.10949/) (2024)
- [CritiqueLLM: Towards an Informative Critique Generation Model for Evaluation of Large Language Model Generation](https://mlsanity.com/paper/2311.18702/) (2023)
- [Learning Planning-based Reasoning by Trajectories Collection and Process Reward Synthesizing](https://mlsanity.com/paper/2402.00658/) (2024)
- [Calibrating the Confidence of Large Language Models by Eliciting Fidelity](https://mlsanity.com/paper/2404.02655/) (2024)
- [Fact-and-Reflection (FaR) Improves Confidence Calibration of Large Language Models](https://mlsanity.com/paper/2402.17124/) (2024)
- [Neuro-Symbolic Data Generation for Math Reasoning](https://mlsanity.com/paper/2412.04857/) (2024)
- [Learning From Correctness Without Prompting Makes LLM Efficient Reasoner](https://mlsanity.com/paper/2403.19094/) (2024)
- [Latent Convergence Modulation in Large Language Models: A Novel Approach to Iterative Contextual Realignment](https://mlsanity.com/paper/2502.06302/) (2025)

## Built on by (most cited first, 6 of 6)

- [Teaching Language Models to Faithfully Express their Uncertainty](https://mlsanity.com/paper/2510.12587/) (2025)
- [Self-Evaluating LLMs for Multi-Step Tasks: Stepwise Confidence Estimation for Failure Detection](https://mlsanity.com/paper/2511.07364/) (2025)
- [Diagnosing Multi-step Reasoning Failures in Black-box LLMs via Stepwise Confidence Attribution](https://mlsanity.com/paper/2605.19228/) (2026)
- [Inference Time Optimization with Confidence Dynamics](https://mlsanity.com/paper/2605.25244/) (2026)
- [When Models Know When They Do Not Know: Calibration, Cascading, and Cleaning](https://mlsanity.com/paper/2601.07965/) (2026)
- [LUMOS: Tracing Parametric Knowledge from Training Data to Behavioral Outputs in LLMs](https://mlsanity.com/paper/2610.02902/) (2026)

## Abstract (arXiv)

While large language models (LLMs) have demonstrated remarkable performance across diverse tasks, they fundamentally lack self-awareness and frequently exhibit overconfidence, assigning high confidence scores to incorrect predictions. Accurate confidence estimation is therefore critical for enhancing the trustworthiness and reliability of LLM-generated outputs. However, existing approaches suffer from coarse-grained scoring mechanisms that fail to provide fine-grained, continuous confidence estimates throughout the generation process. To address these limitations, we introduce FineCE, a novel confidence estimation method that delivers accurate, fine-grained confidence scores during text generation. Specifically, we first develop a comprehensive pipeline for constructing training data that effectively captures the underlying probabilistic distribution of LLM responses, and then train a model to predict confidence scores for arbitrary text sequences in a supervised manner. Furthermore, we propose a Backward Confidence Integration (BCI) strategy that leverages information from the subsequent text to enhance confidence estimation for the current sequence during inference. We also introduce three strategies for identifying optimal positions to perform confidence estimation within the generation process. Extensive experiments on multiple benchmark datasets demonstrate that FineCE consistently outperforms existing classical confidence estimation methods. Our code and all baselines used in the paper are available on GitHub.