mlsanityData to 7 Oct 2026

Teaching Models to Express Their Uncertainty in Words

arXiv 2205.14334 ↗First posted 28 May 2022cs.CL cs.AI cs.LGcited by 731

Not on the map. Why?

Cited by 98 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We show that a GPT-3 model can learn to express uncertainty about its own answers in natural language -- without use of model logits. When given a question, the model generates both an answer and a level of confidence (e.g. "90% confidence" or "high confidence"). These levels map to probabilities that are well calibrated. The model also remains moderately calibrated under distribution shift, and is sensitive to uncertainty in its own answers, rather than imitating human examples. To our knowledge, this is the first time a model has been shown to express calibrated uncertainty about its own answers in natural language. For testing calibration, we introduce the CalibratedMath suite of tasks. We compare the calibration of uncertainty expressed in words ("verbalized probability") to uncertainty extracted from model logits. Both kinds of uncertainty are capable of generalizing calibration under distribution shift. We also provide evidence that GPT-3's ability to generalize calibration depends on pre-trained latent representations that correlate with epistemic uncertainty over its answers.

Key facts. Cited by 731 papers in the map (data through 7 Oct 2026). Builds on 27 of its 36 references that are in the map. First posted on arXiv 28 May 2022.

Builds on

27 of 36 references.

References · 27

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  3. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  4. 2017On Calibration of Modern Neural Networkscited by 4,860
  5. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  6. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  7. 2020Learning to summarize from human feedbackcited by 2,562
  8. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  9. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  10. 2018Deep Anomaly Detection with Outlier Exposurecited by 939
  11. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  12. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 762
15 more
  1. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  2. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  3. 2019Measuring Calibration in Deep Learningcited by 443
  4. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 439
  5. 2021Revisiting the Calibration of Modern Neural Networkscited by 387
  6. 2020Calibrating Deep Neural Networks using Focal Losscited by 353
  7. 2020Calibration of Pre-trained Transformerscited by 307
  8. 2021Recursively Summarizing Books with Human Feedbackcited by 287
  9. 2020Selective Question Answering under Domain Shiftcited by 228
  10. 2020Reducing conversational agents' overconfidence through linguistic calibrationcited by 209
  11. 2017Domain Adaptation for Visual Applications: A Comprehensive Surveycited by 127
  12. 2021Truthful AI: Developing and governing AI that does not liecited by 109
  13. 2021Soft Calibration Objectives for Neural Networkscited by 89
  14. 2015Posterior calibration and exploratory analysis for natural language processing modelscited by 75
  15. 2021Confidence Calibration for Domain Generalization under Covariate Shiftcited by 27

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 9; 2023: 78; 2024: 164; 2025: 189; 2026: 291.

731 papers.

Sort:

By area:

  1. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  2. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  3. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
  4. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
  5. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 788
  6. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
  7. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
  8. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
  9. 2024Measuring short-form factuality in large language modelscited by 345
  10. 2023Robots That Ask For Help: Uncertainty Alignment for Large Language Model Plannerscited by 298
  11. 2023Generating with Confidence: Uncertainty Quantification for Black-box Large Language Modelscited by 286
  12. 2022Prompting GPT-3 To Be Reliablecited by 258
38 more
  1. 2023A Survey of Confidence Estimation and Calibration in Large Language Modelscited by 253
  2. 2023Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questionscited by 249
  3. 2022The Alignment Problem from a Deep Learning Perspectivecited by 246
  4. 2024Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMscited by 246
  5. 2025Why Language Models Hallucinatecited by 205
  6. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  7. 2023Universal Self-Consistency for Large Language Model Generationcited by 198
  8. 2024Don't Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaborationcited by 176
  9. 2023RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Modelscited by 173
  10. 2023Conformal Language Modelingcited by 163
  11. 2023C3: Zero-shot Text-to-SQL with ChatGPTcited by 159
  12. 2024Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similaritiescited by 156
  13. 2023LM vs LM: Detecting Factual Errors via Cross Examinationcited by 155
  14. 2025Reasoning Models Know When They're Right: Probing Hidden States for Self-Verificationcited by 154
  15. 2023Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answeringcited by 150
  16. 2023Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Modelscited by 147
  17. 2023Taken out of context: On measuring situational awareness in LLMscited by 131
  18. 2025AbstentionBench: Reasoning LLMs Fail on Unanswerable Questionscited by 129
  19. 2023Quantifying Uncertainty in Answers from any Language Model and Enhancing their Trustworthinesscited by 121
  20. 2023AI Transparency in the Age of LLMs: A Human-Centered Research Roadmapcited by 119
  21. 2023Chatlaw: A Multi-Agent Legal Assistant based on a Role-Aligned Mixture-of-Experts Architecturecited by 119
  22. 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 116
  23. 2023Do Language Models Know When They're Hallucinating References?cited by 115
  24. 2024A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directionscited by 115
  25. 2023Decomposing Uncertainty for Large Language Models through Input Clarification Ensemblingcited by 113
  26. 2023Explore, Establish, Exploit: Red Teaming Language Models from Scratchcited by 111
  27. 2025Confidence Improves Self-Consistency in LLMscited by 111
  28. 2024What Large Language Models Know and What People Think They Knowcited by 109
  29. 2025Beyond Binary Rewards: Training LMs to Reason About Their Uncertaintycited by 107
  30. 2024Relying on the Unreliable: The Impact of Language Models' Reluctance to Express Uncertaintycited by 106
  31. 2024Taming Overconfidence in LLMs: Reward Calibration in RLHFcited by 105
  32. 2024Looking Inward: Language Models Can Learn About Themselves by Introspectioncited by 105
  33. 2023ToolChain*: Efficient Action Space Navigation in Large Language Models with A* Searchcited by 103
  34. 2024SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationalescited by 102
  35. 2024Large Language Models Must Be Taught to Know What They Don't Knowcited by 99
  36. 2024HaloScope: Harnessing Unlabeled LLM Generations for Hallucination Detectioncited by 96
  37. 2023CoAnnotating: Uncertainty-Guided Work Allocation between Human and Large Language Models for Data Annotationcited by 93
  38. 2023Clarify When Necessary: Resolving Ambiguity Through Interaction with LMscited by 93

50 most cited of 731 All 731 (JSON)