mlsanityData to 14 Oct 2026

Multicalibration for Confidence Scoring in LLMs

arXiv 2404.04689 ↗First posted 6 Apr 2024stat.ML cs.CL cs.LGcited by 46

Not on the map. Why?

Abstract (arXiv)

This paper proposes the use of "multicalibration" to yield interpretable and reliable confidence scores for outputs generated by large language models (LLMs). Multicalibration asks for calibration not just marginally, but simultaneously across various intersecting groupings of the data. We show how to form groupings for prompt/completion pairs that are correlated with the probability of correctness via two techniques: clustering within an embedding space, and "self-annotation" - querying the LLM by asking it various yes-or-no questions about the prompt. We also develop novel variants of multicalibration algorithms that offer performance improvements by reducing their tendency to overfit. Through systematic benchmarking across various question answering datasets and LLMs, we show how our techniques can yield confidence scores that provide substantial improvements in fine-grained measures of both calibration and accuracy compared to existing methods.

Key facts. Cited by 46 papers in the map (data through 14 Oct 2026). Builds on 51 of its 69 references that are in the map. First posted on arXiv 6 Apr 2024.

Builds on

51 of 69 references.

References · 51

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  2. 2017On Calibration of Modern Neural Networkscited by 4,960
  3. 2023Mistral 7Bcited by 3,164
  4. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  5. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  6. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
  7. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
  8. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  9. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  10. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  11. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
  12. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 812
38 more
  1. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 810
  2. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
  3. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 662
  4. 2023A Survey of Hallucination in Large Foundation Modelscited by 368
  5. 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 365
  6. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  7. 2023Robots That Ask For Help: Uncertainty Alignment for Large Language Model Plannerscited by 308
  8. 2023Generating with Confidence: Uncertainty Quantification for Black-box Large Language Modelscited by 293
  9. 2018Multiaccuracy: Black-Box Post-Processing for Fairness in Classificationcited by 222
  10. 2023A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generationcited by 206
  11. 2023LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examplescited by 177
  12. 2021On Hallucination and Predictive Uncertainty in Conditional Language Generationcited by 176
  13. 2023Conformal Language Modelingcited by 170
  14. 2022Domino: Discovering Systematic Errors with Cross-Modal Embeddingscited by 156
  15. 2023Conformal Prediction With Conditional Guaranteescited by 152
  16. 2023Conformal Prediction with Large Language Models for Multi-Choice Question Answeringcited by 139
  17. 2021A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generationcited by 138
  18. 2023Hallucinations in Large Multilingual Translation Modelscited by 129
  19. 2023Calibrated Language Models Must Hallucinatecited by 115
  20. 2023AnglE-optimized Text Embeddingscited by 99
  21. 2023SAC3: Reliable Hallucination Detection in Black-Box Language Models via Semantic-aware Cross-check Consistencycited by 90
  22. 2023Cognitive Mirage: A Review of Hallucinations in Large Language Modelscited by 84
  23. 2021Omnipredictorscited by 74
  24. 2022Batch Multivalid Conformal Predictioncited by 67
  25. 2022Practical Adversarial Multivalid Conformal Predictioncited by 65
  26. 2022Loss Minimization through the Lens of Outcome Indistinguishabilitycited by 62
  27. 2023Uncertainty in Natural Language Generation: From Theory to Applicationscited by 61
  28. 2023Calibrating LLM-Based Evaluatorcited by 53
  29. 2022Low-Degree Multicalibrationcited by 45
  30. 2023High-Dimensional Prediction for Sequential Decision Makingcited by 41
  31. 2023Chainpoll: A high efficacy method for LLM hallucination detectioncited by 39
  32. 2023Halo: Estimation and Reduction of Hallucinations in Open-Source Weak Large Language Modelscited by 36
  33. 2023Multicalibration as Boosting for Regressioncited by 33
  34. 2023Forking Uncertainties: Reliable Prediction and Model Predictive Control with Sequence Models via Conformal Risk Controlcited by 20
  35. 2023Zero-Resource Hallucination Prevention for Large Language Modelscited by 19
  36. 2023Conformal Autoregressive Generation: Beam Search with Coverage Guaranteescited by 16
  37. 2023Uncertainty-aware Language Modeling for Selective Question Answeringcited by 16
  38. 2023Where Does My Model Underperform? A Human Evaluation of Slice Discovery Algorithmscited by 9

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 13; 2025: 16; 2026: 17.

46 papers.

Sort:

By area:

  1. 2024A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directionscited by 121
  2. 2024Large language model validity via enhanced conformal prediction methodscited by 92
  3. 2024LLM-Rubric: A Multidimensional, Calibrated Approach to Automated Evaluation of Natural Language Textscited by 87
  4. 2024Learning to Route LLMs with Confidence Tokenscited by 67
  5. 2025A Survey of Uncertainty Estimation Methods on Large Language Modelscited by 62
  6. 2025From Calibration to Collaboration: LLM Uncertainty Quantification Should Be More Human-Centeredcited by 21
  7. 2024When is Multicalibration Post-Processing Necessary?cited by 18
  8. 2024Cost-Effective Hallucination Detection for LLMscited by 18
  9. 2025Conformal Language Model Reasoning with Coherent Factualitycited by 14
  10. 2024Multi-group Uncertainty Quantification for Long-form Text Generationcited by 13
  11. 2024QA-Calibration of Language Model Confidence Scorescited by 13
  12. 2025Measuring and mitigating overreliance to build human-compatible AIcited by 12
34 more
  1. 2024Self-Healing Machine Learning: A Framework for Autonomous Adaptation in Real-World Environmentscited by 8
  2. 2024A Survey of Calibration Process for Black-Box LLMscited by 8
  3. 2024Calibrating Bayesian Learning via Regularization, Confidence Minimization, and Selective Inferencecited by 7
  4. 2024FactTest: Factuality Testing in Large Language Models with Finite-Sample and Distribution-Free Guaranteescited by 7
  5. 2025Conformal Information Pursuit for Interactively Guiding Large Language Modelscited by 7
  6. 2026ProcCtrlBench: Evaluating Process-Level Defects and Control Preservation in LLM Coding Agentscited by 6
  7. 2025Calibrating LLMs for Text-to-SQL Parsing by Leveraging Sub-clause Frequenciescited by 5
  8. 2025MCGrad: Multicalibration at Web Scalecited by 5
  9. 2025Comparing Uncertainty Measurement and Mitigation Methods for Large Language Models: A Systematic Reviewcited by 4
  10. 2026Prediction-Powered Inference Across Many Tasks for AI Evaluation & Social Science Researchcited by 3
  11. 2025Multiaccuracy and Multicalibration via Proxy Groupscited by 2
  12. 2025DDO: Dual-Decision Optimization for LLM-Based Medical Consultation via Multi-Agent Collaborationcited by 2
  13. 2025Predicting Language Models' Success at Zero-Shot Probabilistic Predictioncited by 2
  14. 2025Generation Space Size: Understanding and Calibrating Open-Endedness of LLM Generationscited by 2
  15. 2025Data Compressibility Quantifies LLM Memorizationcited by 1
  16. 2025Token-Level Marginalization for Multi-Label LLM Classifierscited by 1
  17. 2026Multi-LLM Adaptive Conformal Inference for Reliable LLM Responsescited by 1
  18. 2026"I May Not Have Articulated Myself Clearly": Diagnosing Dynamic Instability in LLM Reasoning at Inference Timecited by 1
  19. 2026On the Convergence of Multicalibration Gradient Boostingcited by 1
  20. 2026Margin-Adaptive Confidence Ranking for Reliable LLM Judgementcited by 1
  21. 2026Conformal Language Modeling via Posterior Samplingcited by 1
  22. 2026Heteroskedastic Signals in Budgeted LLM Verification: Structural Heterogeneity Limits Optimization Gainscited by 1
  23. 2026Localize-Then-Decide Guarantees for LLM Judgmentscited by 1
  24. 2024Latent Feature Mining for Predictive Model Enhancement with Large Language Modelscited by 0
  25. 2025Auditability and the Landscape of Distance to Multicalibrationcited by 0
  26. 2025Enabling Fine-Grained Operating Points for Black-Box LLMscited by 0
  27. 2026Fact-Checking with Large Language Models via Probabilistic Certainty and Consistencycited by 0
  28. 2026Automating the Detection of Requirement Dependencies Using Large Language Modelscited by 0
  29. 2026Fine-grained Approaches for Confidence Calibration of LLMs in Automated Code Revisioncited by 0
  30. 2026Claim-Level Confidence Calibration for Reliable Decision Making with Large Language Modelscited by 0
  31. 2026Structured Claim-Level Discourse Representations for Dense Health Narrativescited by 0
  32. 2026How to Estimate Whether You Have Found Several Needles in a Haystack: Measuring Calibration in Multi-Label Text Classificationcited by 0
  33. 2026Conformal Prediction with Paraphrase-Aware Scoring for LLM Uncertainty Quantificationcited by 0
  34. 2026A CaLLM Judge is a Better Judge: Mitigating Bias in LLM-as-a-judgecited by 0