mlsanityData to 7 Oct 2026

Calibrating Large Language Models Using Their Generations Only

arXiv 2403.05973 ↗First posted 9 Mar 2024cs.CL cs.AI cs.LGcited by 63

Not on the map. Why?

Abstract (arXiv)

As large language models (LLMs) are increasingly deployed in user-facing applications, building trust and maintaining safety by accurately quantifying a model's confidence in its prediction becomes even more important. However, finding effective ways to calibrate LLMs - especially when the only interface to the models is their generated text - remains a challenge. We propose APRICOT (auxiliary prediction of confidence targets): A method to set confidence targets and train an additional model that predicts an LLM's confidence based on its textual input and output alone. This approach has several advantages: It is conceptually simple, does not require access to the target model beyond its output, does not interfere with the language generation, and has a multitude of potential usages, for instance by verbalizing the predicted confidence or adjusting the given answer based on the confidence. We show how our approach performs competitively in terms of calibration error for white-box and black-box LLMs on closed-book question-answering to detect incorrect LLM answers.

Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). Builds on 48 of its 81 references that are in the map. First posted on arXiv 9 Mar 2024.

Builds on

48 of 81 references.

References · 48

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2017On Calibration of Modern Neural Networkscited by 4,860
  5. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  6. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  7. 2017Decoupled Weight Decay Regularizationcited by 1,722
  8. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  9. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
  10. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  11. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
  12. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 788
36 more
  1. 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 777
  2. 2022Teaching Models to Express Their Uncertainty in Wordscited by 731
  3. 2018CoQA: A Conversational Question Answering Challengecited by 674
  4. 2019Quantifying the Carbon Emissions of Machine Learningcited by 516
  5. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 488
  6. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 439
  7. 2023Fine-Grained Human Feedback Gives Better Rewards for Language Model Trainingcited by 399
  8. 2020Calibration of Pre-trained Transformerscited by 307
  9. 2021Deep Deterministic Uncertainty: A Simple Baselinecited by 200
  10. 2020The Next Decade in AI: Four Steps Towards Robust Artificial Intelligencecited by 194
  11. 2020Understanding How Dimension Reduction Tools Work: An Empirical Approach to Deciphering t-SNE, UMAP, TriMAP, and PaCMAP for Data Visualizationcited by 170
  12. 2020Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AIcited by 166
  13. 2021All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Qualitycited by 150
  14. 2020Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasetscited by 137
  15. 2020Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertaintycited by 128
  16. 2023Decomposing Uncertainty for Large Language Models through Input Clarification Ensemblingcited by 113
  17. 2024Relying on the Unreliable: The Impact of Language Models' Reluctance to Express Uncertaintycited by 106
  18. 2023How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questionscited by 82
  19. 2023Uncertainty in Natural Language Generation: From Theory to Applicationscited by 61
  20. 2023Smooth ECE: Principled Reliability Diagrams via Kernel Smoothingcited by 59
  21. 2022A Close Look into the Calibration of Pre-trained Language Modelscited by 57
  22. 2020Trust Issues: Uncertainty Estimation Does Not Enable Reliable OOD Detection On Medical Tabular Datacited by 52
  23. 2022Designing for Responsible Trust in AI Systems: A Communication Perspectivecited by 48
  24. 2024MARS: Meaning-Aware Response Scoring for Uncertainty Estimation in Generative LLMscited by 45
  25. 2019Energy Usage Reports: Environmental awareness as part of algorithmic accountabilitycited by 43
  26. 2021Uncertainty-Aware Machine Translation Evaluationcited by 35
  27. 2019Improving Back-Translation with Uncertainty-based Confidence Estimationcited by 34
  28. 2017An optimal transportation approach for assessing almost stochastic ordercited by 33
  29. 2023ChatGPT may Pass the Bar Exam soon, but has a Long Way to Go for the LexGLUE benchmarkcited by 33
  30. 2020Wat zei je? Detecting Out-of-Distribution Translations with Variational Transformerscited by 31
  31. 2022Exploring Predictive Uncertainty and Calibration in NLP: A Study on the Impact of Method & Data Scarcitycited by 24
  32. 2023A Diachronic Perspective on User Trust in AI under Uncertaintycited by 23
  33. 2024Reconfidencing LLMs from the Grouping Loss Perspectivecited by 11
  34. 2016Exploring Prediction Uncertainty in Machine Translation Quality Estimationcited by 6
  35. 2023Survey on Leveraging Uncertainty Estimation Towards Trustworthy Deep Neural Networks: The Case of Reject Option and Post-training Processingcited by 6
  36. 2023Perspectives on the State and Future of Deep Learning - 2023cited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 14; 2025: 29; 2026: 20.

63 papers.

Sort:

By area:

  1. 2024A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directionscited by 115
  2. 2024Large Language Models Must Be Taught to Know What They Don't Knowcited by 99
  3. 2025Uncertainty Quantification and Confidence Calibration in Large Language Models: A Surveycited by 93
  4. 2025Rewarding Doubt: A Reinforcement Learning Approach to Calibrated Confidence Expression of Large Language Modelscited by 48
  5. 2025Mind the Confidence Gap: Overconfidence, Calibration, and Distractor Effects in Large Language Modelscited by 39
  6. 2025Embodied-R: Collaborative Framework for Activating Embodied Spatial Reasoning in Foundation Models via Reinforcement Learningcited by 37
  7. 2024Teaching Large Language Models to Express Knowledge Boundary from Their Own Signalscited by 32
  8. 2025CDE: Curiosity-Driven Exploration for Efficient Reinforcement Learning in Large Language Modelscited by 28
  9. 2024MAQA: Evaluating Uncertainty Quantification in LLMs Regarding Data Uncertaintycited by 25
  10. 2024Do Not Design, Learn: A Trainable Scoring Function for Uncertainty Estimation in Generative LLMscited by 23
  11. 2024Atomic Calibration of LLMs in Long-Form Generationscited by 23
  12. 2024A Survey on the Honesty of Large Language Modelscited by 21
38 more
  1. 2025From Calibration to Collaboration: LLM Uncertainty Quantification Should Be More Human-Centeredcited by 21
  2. 2025Your Pre-trained LLM is Secretly an Unsupervised Confidence Calibratorcited by 19
  3. 2025LoVeC: Reinforcement Learning for Better Verbalized Confidence in Long-Form Generationscited by 17
  4. 2024LACIE: Listener-Aware Finetuning for Confidence Calibration in Large Language Modelscited by 14
  5. 2025Anthropomimetic Uncertainty: What Verbalized Uncertainty in Language Models is Missingcited by 14
  6. 2024QA-Calibration of Language Model Confidence Scorescited by 13
  7. 2025Can LLMs Predict Their Own Failures? Self-Awareness via Internal Circuitscited by 12
  8. 2024Graph-based Confidence Calibration for Large Language Modelscited by 10
  9. 2024Neural Topic Modeling with Large Language Models in the Loopcited by 10
  10. 2025Dr.LLM: Dynamic Layer Routing in LLMscited by 10
  11. 2024Calibrating Verbalized Probabilities for Large Language Modelscited by 9
  12. 2025Calibration Across Layers: Understanding Calibration Evolution in LLMscited by 9
  13. 2024A Survey of Calibration Process for Black-Box LLMscited by 8
  14. 2026When Agents Say One Thing and Do Another: Validating Elicited Beliefs from LLMscited by 8
  15. 2025Conformal Information Pursuit for Interactively Guiding Large Language Modelscited by 7
  16. 2025Teaching Language Models to Faithfully Express their Uncertaintycited by 7
  17. 2025B-score: Detecting biases in large language models using response historycited by 6
  18. 2025The Consistency Hypothesis in Uncertainty Quantification for Large Language Modelscited by 6
  19. 2025Advancing Event Forecasting through Massive Training of Large Language Models: Challenges, Solutions, and Broader Impactscited by 6
  20. 2025GRACE: A Granular Benchmark for Evaluating Model Calibration against Human Calibrationcited by 5
  21. 2025Can LLMs Refuse Questions They Do Not Know? Measuring Knowledge-Aware Refusal in Factual Taskscited by 5
  22. 2026ProcCtrlBench: Evaluating Process-Level Defects and Control Preservation in LLM Coding Agentscited by 5
  23. 2026Large Language Models Are Overconfident in Their Own Responsescited by 5
  24. 2024Large-scale cloze evaluation reveals that token prediction tasks are neither lexically nor semantically alignedcited by 4
  25. 2025Logits are All We Need to Adapt Closed Modelscited by 4
  26. 2025The Geometry of Creative Variability: How Credal Sets Expose Calibration Gaps in Language Modelscited by 4
  27. 2025Comparing Uncertainty Measurement and Mitigation Methods for Large Language Models: A Systematic Reviewcited by 3
  28. 2025Towards Harmonized Uncertainty Estimation for Large Language Modelscited by 3
  29. 2025ObjexMT: Objective Extraction and Metacognitive Calibration for LLM-as-a-Judge under Multi-Turn Jailbreakscited by 3
  30. 2026Calibration Is Not Enough: Evaluating Confidence Estimation Under Language Variationscited by 3
  31. 2025Bi-directional Model Cascading with Proxy Confidencecited by 2
  32. 2025Predicting Language Models' Success at Zero-Shot Probabilistic Predictioncited by 2
  33. 2025SIMBA UQ: Similarity-Based Aggregation for Uncertainty Quantification in Large Language Modelscited by 2
  34. 2026A Systematic Evaluation of Black-Box Uncertainty Estimation Methods for Large Language Modelscited by 2
  35. 2026Humans and LLMs Diverge on Probabilistic Inferencescited by 1
  36. 2026Shared Doubt: Zero-Shot Cross-Lingual Confidence Estimation for Language Modelscited by 1
  37. 2026CaliDist: Calibrating Large Language Models via Behavioral Robustness to Distractioncited by 1
  38. 2026Speaking in Self-Assessing Tongues: On the Verbalized Confidence of LLMs in Machine Translationcited by 1

50 most cited of 63 All 63 (JSON)