mlsanityData to 14 Oct 2026

Length Optimization in Conformal Prediction

arXiv 2406.18814 ↗First posted 27 Jun 2024stat.ML cs.AI cs.LGcited by 39

Not on the map. Why?

Abstract (arXiv)

Conditional validity and length efficiency are two crucial aspects of conformal prediction (CP). Conditional validity ensures accurate uncertainty quantification for data subpopulations, while proper length efficiency ensures that the prediction sets remain informative. Despite significant efforts to address each of these issues individually, a principled framework that reconciles these two objectives has been missing in the CP literature. In this paper, we develop Conformal Prediction with Length-Optimization (CPL) - a novel and practical framework that constructs prediction sets with (near-) optimal length while ensuring conditional validity under various classes of covariate shifts, including the key cases of marginal and group-conditional coverage. In the infinite sample regime, we provide strong duality results which indicate that CPL achieves conditional validity and length optimality. In the finite sample regime, we show that CPL constructs conditionally valid prediction sets. Our extensive empirical evaluations demonstrate the superior prediction set size performance of CPL compared to state-of-the-art methods across diverse real-world and synthetic datasets in classification, regression, and large language model-based multiple choice question answering. An Implementation of our algorithm can be accessed at the following link: https://github.com/shayankiyani98/CP.

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). Builds on 48 of its 78 references that are in the map. First posted on arXiv 27 Jun 2024.

Builds on

48 of 78 references.

References · 48

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  3. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  4. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  5. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
  6. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  7. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,375
  8. 2016Distribution-Free Predictive Inference For Regressioncited by 668
  9. 2019Conformalized Quantile Regressioncited by 629
  10. 2019Conformal Prediction Under Covariate Shiftcited by 519
  11. 2020Classification with Valid and Adaptive Coveragecited by 396
  12. 2016Least Ambiguous Set-Valued Classifiers with Bounded Error Levelscited by 272
36 more
  1. 2019Predictive inference with the jackknife+cited by 211
  2. 2023Conformal Prediction with Large Language Models for Multi-Choice Question Answeringcited by 139
  3. 2024Language Models with Conformal Factuality Guaranteescited by 131
  4. 2022Safe Planning in Dynamic Environments using Conformal Predictioncited by 119
  5. 2021Localized Conformal Prediction: A Generalized Inference Framework for Conformal Predictioncited by 114
  6. 2021Learning Optimal Conformal Classifierscited by 112
  7. 2023Class-Conditional Conformal Prediction with Many Classescited by 109
  8. 2019Distributional conformal predictioncited by 105
  9. 2024Large language model validity via enhanced conformal prediction methodscited by 92
  10. 2020Robust Validation: Confident Predictions Even When Distributions Shiftcited by 82
  11. 2021Conformal Prediction using Conditional Histogramscited by 77
  12. 2020CD-split and HPD-split: efficient conformal regions in high dimensionscited by 76
  13. 2022Batch Multivalid Conformal Predictioncited by 67
  14. 2019Flexible distribution-free conditional predictive bands using density estimatorscited by 64
  15. 2019PAC Confidence Sets for Deep Neural Networks via Calibrated Predictioncited by 60
  16. 2014Regression Conformal Prediction with Nearest Neighbourscited by 59
  17. 2022Improving Expert Predictions with Conformal Predictioncited by 54
  18. 2021Selection and Aggregation of Conformal Prediction Setscited by 49
  19. 2021Improving Conditional Coverage via Orthogonal Quantile Regressioncited by 48
  20. 2023Conformal prediction with local weights: randomization enables local guaranteescited by 48
  21. 2023High-Dimensional Prediction for Sequential Decision Makingcited by 41
  22. 2024Conformal Prediction Sets Improve Human Decision Makingcited by 35
  23. 2022On the Utility of Prediction Sets in Human-AI Teamscited by 34
  24. 2023Conformal Prediction Regions for Time Series using Linear Complementarity Programmingcited by 33
  25. 2024Boosted Conformal Prediction Intervalscited by 27
  26. 2022Efficient and Differentiable Conformal Prediction with General Function Classescited by 26
  27. 2023Designing Decision Support Systems Using Counterfactual Prediction Setscited by 26
  28. 2024Conformal Prediction with Learned Featurescited by 24
  29. 2023PAC Prediction Sets Under Label Shiftcited by 21
  30. 2023Adaptive Conformal Prediction by Reweighting Nonconformity Scorecited by 16
  31. 2022Conformal Frequency Estimation using Discrete Sketched Data with Coverage for Distinct Queriescited by 10
  32. 2023Federated Inference with Reliable Uncertainty Quantification over Wireless Channels via Conformal Predictioncited by 10
  33. 2024Generalization and Informativeness of Conformal Predictioncited by 9
  34. 2022Prediction Sets for High-Dimensional Mixture of Experts Modelscited by 8
  35. 2023Adaptive Conformal Regression with Jackknife+ Rescaled Scorescited by 8
  36. 2021MD-split+: Practical Local Conformal Inference in High Dimensionscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 23; 2026: 13.

39 papers.

Sort:

  1. 2025Decision Theoretic Foundations for Conformal Prediction: Optimal Uncertainty Quantification for Risk-Averse Agentscited by 40
  2. 2025Minimum Volume Conformal Sets for Multivariate Regressioncited by 27
  3. 2024Prune 'n Predict: Optimizing LLM Decision-making with Conformal Predictioncited by 18
  4. 2025Backward Conformal Predictioncited by 14
  5. 2025Conformal Inference under High-Dimensional Covariate Shifts via Likelihood-Ratio Regularizationcited by 12
  6. 2024Conformal Risk Minimization with Variance Reductioncited by 11
  7. 2025Volume Optimality in Conformal Prediction with Structured Prediction Setscited by 10
  8. 2025Conformal Risk Control for Semantic Uncertainty Quantification in Computed Tomographycited by 8
  9. 2025Direct Prediction Set Minimization via Bilevel Conformal Classifier Trainingcited by 8
  10. 2025Parametric Scaling Law of Tuning Bias in Conformal Predictioncited by 7
  11. 2025Conditional Conformal Risk Adaptationcited by 7
  12. 2025Conformal Information Pursuit for Interactively Guiding Large Language Modelscited by 7
27 more
  1. 2025Conformal Prediction for Long-Tailed Classificationcited by 7
  2. 2025Human-AI Collaborative Uncertainty Quantificationcited by 7
  3. 2025Conformal Prediction Beyond the Seen: A Missing Mass Perspective for Uncertainty Quantification in Generative Modelscited by 6
  4. 2025Domain-Shift-Aware Conformal Prediction for Large Language Modelscited by 6
  5. 2025Non-exchangeable Conformal Prediction with Optimal Transport: Tackling Distribution Shifts with Unlabeled Datacited by 5
  6. 2025Conformal Prediction in The Loop: A Feedback-Based Uncertainty Model for Trajectory Optimizationcited by 3
  7. 2025Singleton-Optimized Conformal Predictioncited by 2
  8. 2025Non-Asymptotic Analysis of Efficiency in Conformalized Regressioncited by 2
  9. 2025CLAPS: Aleatoric-Epistemic Scaling via Last-Layer Laplace for Conformal Regressioncited by 2
  10. 2026Co-optimization for Adaptive Conformal Predictioncited by 2
  11. 2026Conformal Risk-Averse Decision Making with Action Conditional Guaranteecited by 2
  12. 2025Dual-Splitting Conformal Prediction for Multi-Step Time Series Forecastingcited by 1
  13. 2026Learning Polyhedral Conformal Sets for Robust Optimizationcited by 1
  14. 2026Empirical Bayes Conformal Prediction for Vision and Language Modelscited by 1
  15. 2026On Optimal Data Splitting for Split Conformal Predictioncited by 1
  16. 2024Data-light Uncertainty Set Merging with Admissibilitycited by 0
  17. 2025Feature Fitted Online Conformal Prediction for Deep Time Series Forecasting Modelcited by 0
  18. 2025Smoothing-Based Conformal Prediction for Balancing Efficiency and Interpretabilitycited by 0
  19. 2025Conformal Correction for Efficiency May be at Odds with Entropycited by 0
  20. 2026Questioning the Coverage-Length Metric in Conformal Prediction: When Shorter Intervals Are Not Bettercited by 0
  21. 2026AdaptNC: Adaptive Nonconformity Scores for Conformal Prediction under Distribution Shiftcited by 0
  22. 2026Compact Conformal Subgraphscited by 0
  23. 2026Conformal Margin Risk Minimization: An Envelope Framework for Robust Learning under Label Noisecited by 0
  24. 2026Geometry of tail allocation in conformal prediction intervalscited by 0
  25. 2026Stable Localized Conformal Prediction via Transductioncited by 0
  26. 2026Robust Bayes-Assisted Conformal Predictioncited by 0
  27. 2026Valid and Efficient Split Conformal Regression for Time Seriescited by 0