mlsanityData to 7 Oct 2026

Conformal Prediction with Large Language Models for Multi-Choice Question Answering

arXiv 2305.18404 ↗First posted 28 May 2023cs.CL cs.LG stat.MLcited by 131

Not on the map. Why?

Abstract (arXiv)

As large language models continue to be widely developed, robust uncertainty quantification techniques will become crucial for their safe deployment in high-stakes scenarios. In this work, we explore how conformal prediction can be used to provide uncertainty quantification in language models for the specific task of multiple-choice question-answering. We find that the uncertainty estimates from conformal prediction are tightly correlated with prediction accuracy. This observation can be useful for downstream applications such as selective classification and filtering out low-quality predictions. We also investigate the exchangeability assumption required by conformal prediction to out-of-subject questions, which may be a more realistic scenario for many practical applications. Our work contributes towards more trustworthy and reliable usage of large language models in safety-critical situations, where robust guarantees of error rate are required.

Key facts. Cited by 131 papers in the map (data through 7 Oct 2026). Builds on 19 of its 36 references that are in the map. First posted on arXiv 28 May 2023.

Builds on

19 of 36 references.

References · 19

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  5. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  6. 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 777
  7. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 439
  8. 2020Uncertainty Sets for Image Classifiers using Conformal Predictioncited by 365
  9. 2016Least Ambiguous Set-Valued Classifiers with Bounded Error Levelscited by 270
  10. 2021Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Controlcited by 256
  11. 2020Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertaintycited by 128
  12. 2013Learning by Transductioncited by 120
7 more
  1. 2021Fair Conformal Predictors for Applications in Medical Imagingcited by 74
  2. 2022Improving Trustworthiness of AI Disease Severity Rating in Medical Imaging with Ordinal Conformal Prediction Setscited by 38
  3. 2022On the Utility of Prediction Sets in Human-AI Teamscited by 34
  4. 2020Empirical Frequentist Coverage of Deep Learning Uncertainty Quantification Procedurescited by 19
  5. 2021Distribution-Free Federated Learning with Conformal Predictionscited by 16
  6. 2022Three Applications of Conformal Prediction for Rating Breast Density in Mammographycited by 6
  7. 2022Towards Reliable Zero Shot Classification in Self-Supervised Models with Conformal Predictioncited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 9; 2024: 40; 2025: 38; 2026: 44.

131 papers.

Sort:

By area:

  1. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
  2. 2023Robots That Ask For Help: Uncertainty Alignment for Large Language Model Plannerscited by 298
  3. 2024Retrieval-Augmented Generation with Graphs (GraphRAG)cited by 170
  4. 2024Language Models with Conformal Factuality Guaranteescited by 128
  5. 2024Benchmarking LLMs via Uncertainty Quantificationcited by 124
  6. 2024A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directionscited by 115
  7. 2025Uncertainty Quantification and Confidence Calibration in Large Language Models: A Surveycited by 93
  8. 2024Large language model validity via enhanced conformal prediction methodscited by 91
  9. 2024Uncertainty Estimation and Quantification for LLMs: A Simple Supervised Approachcited by 73
  10. 2024API Is Enough: Conformal Prediction for Large Language Models Without Logit-Accesscited by 68
  11. 2024ConU: Conformal Uncertainty in Large Language Models with Correctness Coverage Guaranteescited by 62
  12. 2025A Survey of Uncertainty Estimation Methods on Large Language Modelscited by 57
38 more
  1. 2023Conformal Prediction for Deep Classifier via Label Rankingcited by 52
  2. 2024Conformal Prediction for Natural Language Processing: A Surveycited by 52
  3. 2024Conformal Alignment: Knowing When to Trust Foundation Models with Guaranteescited by 50
  4. 2024Calibrating the Confidence of Large Language Models by Eliciting Fidelitycited by 47
  5. 2024Multicalibration for Confidence Scoring in LLMscited by 44
  6. 2024Length Optimization in Conformal Predictioncited by 39
  7. 2024Knowledge Boundary of Large Language Models: A Surveycited by 36
  8. 2024Conformal Prediction Sets Improve Human Decision Makingcited by 35
  9. 2025Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Surveycited by 29
  10. 2024Uncertainty-Aware Evaluation for Vision-Language Modelscited by 28
  11. 2024Cycles of Thought: Measuring LLM Confidence through Stable Explanationscited by 27
  12. 2025Analyzing Uncertainty of LLM-as-a-Judge: Interval Evaluations with Conformal Predictioncited by 26
  13. 2024Conformal Prediction: A Data Perspectivecited by 25
  14. 2023TRAQ: Trustworthy Retrieval Augmented Question Answering via Conformal Predictioncited by 23
  15. 2023Conformal Temporal Logic Planning using Large Language Modelscited by 21
  16. 2024Non-Exchangeable Conformal Language Generation with Nearest Neighborscited by 21
  17. 2025Multivariate Conformal Prediction using Optimal Transportcited by 21
  18. 2024Trustworthy Classification through Rank-Based Conformal Prediction Setscited by 20
  19. 2025Conformal Linguistic Calibration: Trading-off between Factuality and Specificitycited by 20
  20. 2025CP-Router: An Uncertainty-Aware Router Between LLM and LRMcited by 19
  21. 2024Probabilistically Correct Language-based Multi-Robot Planning using Conformal Predictioncited by 18
  22. 2024Sample then Identify: A General Framework for Risk Control and Assessment in Multimodal Large Language Modelscited by 18
  23. 2023Prompt Risk Control: A Rigorous Framework for Responsible Deployment of Large Language Modelscited by 17
  24. 2024Hallucination Detection in Foundation Models for Decision-Making: A Flexible Definition and Review of the State of the Artcited by 17
  25. 2024Efficient Non-Parametric Uncertainty Quantification for Black-Box Large Language Models and Decision Planningcited by 16
  26. 2024Towards Rationality in Language and Multimodal Agents: A Surveycited by 15
  27. 2024Prune 'n Predict: Optimizing LLM Decision-making with Conformal Predictioncited by 15
  28. 2026Uncertainty Quantification in LLM Agents: Foundations, Emerging Challenges, and Opportunitiescited by 15
  29. 2025Conformal Language Model Reasoning with Coherent Factualitycited by 14
  30. 2024Multi-group Uncertainty Quantification for Long-form Text Generationcited by 13
  31. 2025Backward Conformal Predictioncited by 13
  32. 2026Knowing When to Abstain: Medical LLMs Under Clinical Uncertaintycited by 13
  33. 2024Does confidence calibration improve conformal prediction?cited by 12
  34. 2025Kandinsky Conformal Prediction: Beyond Class- and Covariate-Conditional Coveragecited by 12
  35. 2025Seeing and Reasoning with Confidence: Supercharging Multimodal LLMs with an Uncertainty-Aware Agentic Frameworkcited by 12
  36. 2025Conformal Inference under High-Dimensional Covariate Shifts via Likelihood-Ratio Regularizationcited by 11
  37. 2025Object-Level Verbalized Confidence Calibration in Vision-Language Models via Semantic Perturbationcited by 11
  38. 2023Estimating Uncertainty in Multimodal Foundation Models using Public Internet Datacited by 10

50 most cited of 131 All 131 (JSON)