mlsanityData to 7 Oct 2026

ConU: Conformal Uncertainty in Large Language Models with Correctness Coverage Guarantees

arXiv 2407.00499 ↗First posted 29 Jun 2024cs.CL cs.AI cs.LGcited by 62

Not on the map. Why?

Abstract (arXiv)

Uncertainty quantification (UQ) in natural language generation (NLG) tasks remains an open challenge, exacerbated by the closed-source nature of the latest large language models (LLMs). This study investigates applying conformal prediction (CP), which can transform any heuristic uncertainty notion into rigorous prediction sets, to black-box LLMs in open-ended NLG tasks. We introduce a novel uncertainty measure based on self-consistency theory, and then develop a conformal uncertainty criterion by integrating the uncertainty condition aligned with correctness into the CP algorithm. Empirical evaluations indicate that our uncertainty measure outperforms prior state-of-the-art methods. Furthermore, we achieve strict control over the correctness coverage rate utilizing 7 popular LLMs on 4 free-form NLG datasets, spanning general-purpose and medical scenarios. Additionally, the calibrated prediction sets with small size further highlights the efficiency of our method in providing trustworthy guarantees for practical open-ended NLG applications.

Key facts. Cited by 62 papers in the map (data through 7 Oct 2026). Builds on 24 of its 42 references that are in the map. First posted on arXiv 29 Jun 2024.

Builds on

24 of 42 references.

References · 24

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  7. 2023Mistral 7Bcited by 3,065
  8. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  9. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  10. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  11. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
  12. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 788
12 more
  1. 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 777
  2. 2022Teaching Models to Express Their Uncertainty in Wordscited by 731
  3. 2018CoQA: A Conversational Question Answering Challengecited by 674
  4. 2023Conformal Prediction with Large Language Models for Multi-Choice Question Answeringcited by 131
  5. 2024Language Models with Conformal Factuality Guaranteescited by 128
  6. 2024Benchmarking LLMs via Uncertainty Quantificationcited by 124
  7. 2024GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluationscited by 118
  8. 2024API Is Enough: Conformal Prediction for Large Language Models Without Logit-Accesscited by 68
  9. 2024Mitigating LLM Hallucinations via Conformal Abstentioncited by 56
  10. 2024Conformal Prediction for Natural Language Processing: A Surveycited by 52
  11. 2024Characterizing Truthfulness in Large Language Model Generations with Local Intrinsic Dimensioncited by 45
  12. 2023Equal Opportunity of Coverage in Fair Regressioncited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 33; 2026: 25.

62 papers.

Sort:

By area:

  1. 2024A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directionscited by 115
  2. 2025Uncertainty Quantification and Confidence Calibration in Large Language Models: A Surveycited by 93
  3. 2024LLaVA Steering: Visual Instruction Tuning with 500x Fewer Parameters through Modality Linear Representation-Steeringcited by 50
  4. 2025ASCD: Attention-Steerable Contrastive Decoding for Reducing Hallucination in MLLMcited by 36
  5. 2025Analyzing Uncertainty of LLM-as-a-Judge: Interval Evaluations with Conformal Predictioncited by 26
  6. 2025Self-Evolving Multi-Agent Systems via Textual Backpropagationcited by 21
  7. 2025CP-Router: An Uncertainty-Aware Router Between LLM and LRMcited by 19
  8. 2024Sample then Identify: A General Framework for Risk Control and Assessment in Multimodal Large Language Modelscited by 18
  9. 2026SCOPE: Selective Conformal Optimized Pairwise LLM Judgingcited by 12
  10. 2024Reversal of Thought: Enhancing Large Language Models with Preference-Guided Reverse Reasoning Warm-upcited by 10
  11. 2025Conformal Prediction Sets for Deep Generative Models via Reduction to Conformal Regressioncited by 10
  12. 2025Simple Yet Effective: An Information-Theoretic Approach to Multi-LLM Uncertainty Quantificationcited by 6
38 more
  1. 2025Domain-Shift-Aware Conformal Prediction for Large Language Modelscited by 6
  2. 2025COPU: Conformal Prediction for Uncertainty Quantification in Natural Language Generationcited by 5
  3. 2025TECP: Token-Entropy Conformal Prediction for LLMscited by 5
  4. 2025Selective Risk Certification for LLM Outputs via Information-Lift Statistics: PAC-Bayes, Robustness, and Skeleton Designcited by 5
  5. 2025Uncertainty Quantification for Retrieval-Augmented Reasoningcited by 5
  6. 2026Geometry-Calibrated Conformal Abstention for Language Modelscited by 5
  7. 2025Conformal Lesion Segmentation for 3D Medical Imagescited by 4
  8. 2025CritiCal: Can Critique Help LLM Uncertainty or Confidence Calibration?cited by 4
  9. 2025LEC: Linear Expectation Constraints for Selection-Conditioned Risk Control in Selective Prediction and Routing Systemscited by 4
  10. 2025Data-Driven Calibration of Prediction Sets in Large Vision-Language Models Based on Inductive Conformal Predictioncited by 3
  11. 2025Comparing Uncertainty Measurement and Mitigation Methods for Large Language Models: A Systematic Reviewcited by 3
  12. 2025Calibrated Predictive Lower Bounds on Time-to-Unsafe-Sampling in LLMscited by 3
  13. 2025Quantifying Uncertainty in Natural Language Explanations of Large Language Models for Question Answeringcited by 3
  14. 2026Two Calls, Two Moments, and the Vote-Accuracy Curve of Repeated LLM Inferencecited by 3
  15. 2026EchoRL: Reinforcement Learning via Rollout Echoingcited by 3
  16. 2025Coverage-Guaranteed Speech Emotion Recognition via Calibrated Uncertainty-Adaptive Prediction Setscited by 2
  17. 2025Statistical Management of the False Discovery Rate in Medical Instance Segmentation Based on Conformal Risk Controlcited by 2
  18. 2025Conformal Segmentation in Industrial Surface Defect Detection with Statistical Guaranteescited by 2
  19. 2025High Accuracy, Less Talk (HALT): Reliable LLMs through Capability-Aligned Finetuningcited by 2
  20. 2025Argument-Based Consistency in Toxicity Explanations of LLMscited by 2
  21. 2025A Hierarchical Imprecise Probability Approach to Reliability Assessment of Large Language Modelscited by 2
  22. 2026Black-Box Reliability Certification for AI Agents via Self-Consistency Sampling and Conformal Calibrationcited by 2
  23. 2026BalanceRAG: Joint Risk Calibration for Cascaded Retrieval-Augmented Generationcited by 2
  24. 2026MM-Snowball: Evaluating and Mitigating Hallucination Snowballing in Multimodal Multi-Turn Dialoguecited by 2
  25. 2025Correctness Coverage Evaluation for Medical Multiple-Choice Question Answering Based on the Enhanced Conformal Prediction Frameworkcited by 1
  26. 2025Conformal Prediction for Indoor Positioning with Correctness Coverage Guaranteescited by 1
  27. 2025ALIEN: Aligned Entropy Head for Improving Uncertainty Estimation of LLMscited by 1
  28. 2025Conformal Sets in Multiple-Choice Question Answering under Black-Box Settings with Provable Coverage Guaranteescited by 1
  29. 2026CAOS: Conformal Aggregation of One-Shot Predictorscited by 1
  30. 2026From Passive Metric to Active Signal: The Evolving Role of Uncertainty Quantification in Large Language Modelscited by 1
  31. 2026Semantic Token Clustering for Efficient Uncertainty Quantification in Large Language Modelscited by 1
  32. 2026Beyond Surface Statistics: Robust Conformal Prediction for LLMs via Internal Representationscited by 1
  33. 2026Differentiable Conformal Training for LLM Reasoning Factualitycited by 1
  34. 2026Process Supervision of Confidence Margin for Calibrated LLM Reasoningcited by 1
  35. 2026When Can Voting Help, Hurt, or Change Course? Exact Structure of Binary Test-Time Aggregationcited by 1
  36. 2026Conformal Certification of Reasoning Trace Prefixescited by 1
  37. 2026CARE: A Conformal Safety Layer for Medical Summarizationcited by 1
  38. 2026Uncertainty-Aware Abstention in Large Language Models with Provable Alignment Guaranteescited by 1

50 most cited of 62 All 62 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026The Origins of Stochasticity: Comprehensive Investigations on Uncertainty Quantification for Large Language Models
  2. 2024Rethinking Uncertainty Estimation in LLMs: A Principled Single-Sequence Measure
  3. 2025Robust Uncertainty Quantification for Self-Evolving Large Language Models via Continual Domain Pretraining
  4. 2026Robust Conformal Consensus: Multi-Agent LLM-as-a-Judge Interval Evaluation with Conformal Prediction
  5. 2025Revisiting Uncertainty Estimation and Calibration of Large Language Models
  6. 2023Quantifying Uncertainty in Natural Language Explanations of Large Language Models
  7. 2024LoGU: Long-form Generation with Uncertainty Expressions
  8. 2026Dissecting Training-Free Uncertainty Estimation in Multimodal Large Language Models