# Quantifying Uncertainty in Natural Language Explanations of Large Language Models

arXiv: https://arxiv.org/abs/2311.03533 · HTML: https://mlsanity.com/paper/2311.03533/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 51 references that are in the map.
- First posted on arXiv 6 Nov 2023.

## Builds on (most cited first, 30 of 36)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.04388/) (2023)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs](https://mlsanity.com/paper/2306.13063/) (2023)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [Are NLP Models really able to Solve Simple Math Word Problems?](https://mlsanity.com/paper/2103.07191/) (2021)
- [Teaching Models to Express Their Uncertainty in Words](https://mlsanity.com/paper/2205.14334/) (2022)
- [Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies](https://mlsanity.com/paper/2101.02235/) (2021)
- [Measuring Faithfulness in Chain-of-Thought Reasoning](https://mlsanity.com/paper/2307.13702/) (2023)
- [Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment](https://mlsanity.com/paper/1907.11932/) (2019)
- [Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?](https://mlsanity.com/paper/2004.03685/) (2020)
- [How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering](https://mlsanity.com/paper/2012.00955/) (2020)
- [The (Un)reliability of saliency methods](https://mlsanity.com/paper/1711.00867/) (2017)
- [A Diverse Corpus for Evaluating and Developing English Math Word Problem Solvers](https://mlsanity.com/paper/2106.15772/) (2021)
- [Faithful Chain-of-Thought Reasoning](https://mlsanity.com/paper/2301.13379/) (2023)
- [Challenges and Applications of Large Language Models](https://mlsanity.com/paper/2307.10169/) (2023)
- [Understanding Neural Networks through Representation Erasure](https://mlsanity.com/paper/1612.08220/) (2016)
- [Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models](https://mlsanity.com/paper/2101.00288/) (2021)
- [Visualizing and Understanding Neural Models in NLP](https://mlsanity.com/paper/1506.01066/) (2015)

## Built on by (most cited first, 30 of 32)

- [Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty](https://mlsanity.com/paper/2507.16806/) (2025)
- [Rethinking Interpretability in the Era of Large Language Models](https://mlsanity.com/paper/2402.01761/) (2024)
- [What Large Language Models Know and What People Think They Know](https://mlsanity.com/paper/2401.13835/) (2024)
- [On Verbalized Confidence Scores for LLMs](https://mlsanity.com/paper/2412.14737/) (2024)
- [Understanding the Uncertainty of LLM Explanations: A Perspective Based on Reasoning Topology](https://mlsanity.com/paper/2502.17026/) (2025)
- [Conformal Linguistic Calibration: Trading-off between Factuality and Specificity](https://mlsanity.com/paper/2502.19110/) (2025)
- [Uncertainty Quantification in LLM Agents: Foundations, Emerging Challenges, and Opportunities](https://mlsanity.com/paper/2602.05073/) (2026)
- [MUR: Momentum Uncertainty guided Reasoning for Large Language Models](https://mlsanity.com/paper/2507.14958/) (2025)
- [The challenge of uncertainty quantification of large language models in medicine](https://mlsanity.com/paper/2504.05278/) (2025)
- [Anthropomimetic Uncertainty: What Verbalized Uncertainty in Language Models is Missing](https://mlsanity.com/paper/2507.10587/) (2025)
- [Uncertainty Quantification in Large Language Models Through Convex Hull Analysis](https://mlsanity.com/paper/2406.19712/) (2024)
- [Standard Benchmarks Fail -- Auditing LLM Agents in Finance Must Prioritize Risk](https://mlsanity.com/paper/2502.15865/) (2025)
- [Robust Answers, Fragile Logic: Probing the Decoupling Hypothesis in LLM Reasoning](https://mlsanity.com/paper/2505.17406/) (2025)
- [Always Tell Me The Odds: Fine-grained Conditional Probability Estimation](https://mlsanity.com/paper/2505.01595/) (2025)
- [RACER: An LLM-powered Methodology for Scalable Analysis of Semi-structured Mental Health Interviews](https://mlsanity.com/paper/2402.02656/) (2024)
- [Securing the Future of GenAI: Policy and Technology](https://mlsanity.com/paper/2407.12999/) (2024)
- [Comparing Uncertainty Measurement and Mitigation Methods for Large Language Models: A Systematic Review](https://mlsanity.com/paper/2504.18346/) (2025)
- [REMI: A Novel Causal Schema Memory Architecture for Personalized Lifestyle Recommendation Agents](https://mlsanity.com/paper/2509.06269/) (2025)
- [Humans overrely on overconfident language models, across languages](https://mlsanity.com/paper/2507.06306/) (2025)
- [Explainability of Large Language Models: Opportunities and Challenges toward Generating Trustworthy Explanations](https://mlsanity.com/paper/2510.17256/) (2025)
- [Finding the Cracks: Improving LLMs Reasoning with Paraphrastic Probing and Consistency Verification](https://mlsanity.com/paper/2602.11361/) (2026)
- [Can LLMs Use Linguistic Uncertainty Markers to Reliably Reflect Intrinsic Confidence?](https://mlsanity.com/paper/2605.28778/) (2026)
- [A Systematic Evaluation of Black-Box Uncertainty Estimation Methods for Large Language Models](https://mlsanity.com/paper/2606.19868/) (2026)
- [Optimizing Chain-of-Thought Confidence via Topological and Dirichlet Risk Analysis](https://mlsanity.com/paper/2511.06437/) (2025)
- [Performance Gains of LLMs With Humans in a World of LLMs Versus Humans](https://mlsanity.com/paper/2505.08902/) (2025)
- [LAMP: Extracting Local Decision Surfaces From Large Language Models](https://mlsanity.com/paper/2505.11772/) (2025)
- [Towards Reliable, Uncertainty-Aware Alignment](https://mlsanity.com/paper/2507.15906/) (2025)
- [Can LLMs extract scientific consensus? A case study in high-temperature superconductivity](https://mlsanity.com/paper/2606.07570/) (2026)
- [The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing](https://mlsanity.com/paper/2606.25108/) (2026)
- [Consistent but Miscalibrated: Evaluating LLM Limitations for Risk Communication in Natural Language](https://mlsanity.com/paper/2607.03882/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) are increasingly used as powerful tools for several high-stakes natural language processing (NLP) applications. Recent prompting works claim to elicit intermediate reasoning steps and key tokens that serve as proxy explanations for LLM predictions. However, there is no certainty whether these explanations are reliable and reflect the LLMs behavior. In this work, we make one of the first attempts at quantifying the uncertainty in explanations of LLMs. To this end, we propose two novel metrics -- textitVerbalized Uncertainty and textitProbing Uncertainty -- to quantify the uncertainty of generated explanations. While verbalized uncertainty involves prompting the LLM to express its confidence in its explanations, probing uncertainty leverages sample and model perturbations as a means to quantify the uncertainty. Our empirical analysis of benchmark datasets reveals that verbalized uncertainty is not a reliable estimate of explanation confidence. Further, we show that the probing uncertainty estimates are correlated with the faithfulness of an explanation, with lower uncertainty corresponding to explanations with higher faithfulness. Our study provides insights into the challenges and opportunities of quantifying uncertainty in LLM explanations, contributing to the broader discussion of the trustworthiness of foundation models.