mlsanityData to 14 Oct 2026

Domain-Shift-Aware Conformal Prediction for Large Language Models

arXiv 2510.05566 ↗First posted 7 Oct 2025stat.ML cs.AI cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

Large language models have achieved impressive performance across diverse tasks. However, their tendency to produce overconfident and factually incorrect outputs, known as hallucinations, poses risks in real-world applications. Conformal prediction provides finite-sample, distribution-free coverage guarantees, but standard conformal prediction breaks down under domain shift, often leading to under-coverage and unreliable prediction sets. We propose a new framework called Domain-Shift-Aware Conformal Prediction (DS-CP). Our framework adapts conformal prediction to large language models under domain shift, by systematically reweighting calibration samples based on their proximity to the test prompt, thereby preserving validity while enhancing adaptivity. Our theoretical analysis and experiments on the MMLU benchmark demonstrate that the proposed method delivers more reliable coverage than standard conformal prediction, especially under substantial distribution shifts, while maintaining efficiency. This provides a practical step toward trustworthy uncertainty quantification for large language models in real-world deployment.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 32 of its 40 references that are in the map. First posted on arXiv 7 Oct 2025.

Builds on

32 of 40 references.

References · 32

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  4. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  5. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  6. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  7. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,515
  8. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
  9. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
  10. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,102
  11. 2020In Search of Lost Domain Generalizationcited by 1,054
  12. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 900
20 more
  1. 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 792
  2. 2021A Survey of Uncertainty in Deep Neural Networkscited by 680
  3. 2016Distribution-Free Predictive Inference For Regressioncited by 668
  4. 2019Conformalized Quantile Regressioncited by 629
  5. 2019Conformal Prediction Under Covariate Shiftcited by 519
  6. 2020Uncertainty Sets for Image Classifiers using Conformal Predictioncited by 366
  7. 2022Conformal prediction beyond exchangeabilitycited by 355
  8. 2023Robots That Ask For Help: Uncertainty Alignment for Large Language Model Plannerscited by 308
  9. 2023Conformal Language Modelingcited by 170
  10. 2023Conformal Prediction with Large Language Models for Multi-Choice Question Answeringcited by 139
  11. 2024Language Models with Conformal Factuality Guaranteescited by 131
  12. 2024Benchmarking LLMs via Uncertainty Quantificationcited by 126
  13. 2024A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directionscited by 121
  14. 2024Large language model validity via enhanced conformal prediction methodscited by 92
  15. 2024API Is Enough: Conformal Prediction for Large Language Models Without Logit-Accesscited by 69
  16. 2024ConU: Conformal Uncertainty in Large Language Models with Correctness Coverage Guaranteescited by 65
  17. 2024Length Optimization in Conformal Predictioncited by 39
  18. 2024Non-Exchangeable Conformal Language Generation with Nearest Neighborscited by 21
  19. 2024Prune 'n Predict: Optimizing LLM Decision-making with Conformal Predictioncited by 18
  20. 2025Projection Pursuit Density Ratio Estimationcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 5.

6 papers.

  1. 2026CAOS: Conformal Aggregation of One-Shot Predictorscited by 1
  2. 2026Adaptive Conformal Prediction for Improving Factuality of Generations by Large Language Modelscited by 1
  3. 2026Beyond Surface Statistics: Robust Conformal Prediction for LLMs via Internal Representationscited by 1
  4. 2025CoFineLLM: Conformal Finetuning of LLMs for Language-Instructed Robot Planningcited by 0
  5. 2026Continual Calibration: Coverage Can Collapse Before Accuracy in Lifelong LLM Fine-Tuningcited by 0
  6. 2026Can VLMs Reliably Assess Sidewalk Accessibility Attributes from Pedestrian-Level Imagery?cited by 0