# Length Optimization in Conformal Prediction

arXiv: https://arxiv.org/abs/2406.18814 · HTML: https://mlsanity.com/paper/2406.18814/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 78 references that are in the map.
- First posted on arXiv 27 Jun 2024.

## Builds on (most cited first, 30 of 48)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [PIQA: Reasoning about Physical Commonsense in Natural Language](https://mlsanity.com/paper/1911.11641/) (2019)
- [Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering](https://mlsanity.com/paper/1809.02789/) (2018)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Distribution-Free Predictive Inference For Regression](https://mlsanity.com/paper/1604.04173/) (2016)
- [Conformalized Quantile Regression](https://mlsanity.com/paper/1905.03222/) (2019)
- [Conformal Prediction Under Covariate Shift](https://mlsanity.com/paper/1904.06019/) (2019)
- [Classification with Valid and Adaptive Coverage](https://mlsanity.com/paper/2006.02544/) (2020)
- [Least Ambiguous Set-Valued Classifiers with Bounded Error Levels](https://mlsanity.com/paper/1609.00451/) (2016)
- [Predictive inference with the jackknife+](https://mlsanity.com/paper/1905.02928/) (2019)
- [Conformal Prediction with Large Language Models for Multi-Choice Question Answering](https://mlsanity.com/paper/2305.18404/) (2023)
- [Language Models with Conformal Factuality Guarantees](https://mlsanity.com/paper/2402.10978/) (2024)
- [Safe Planning in Dynamic Environments using Conformal Prediction](https://mlsanity.com/paper/2210.10254/) (2022)
- [Localized Conformal Prediction: A Generalized Inference Framework for Conformal Prediction](https://mlsanity.com/paper/2106.08460/) (2021)
- [Learning Optimal Conformal Classifiers](https://mlsanity.com/paper/2110.09192/) (2021)
- [Class-Conditional Conformal Prediction with Many Classes](https://mlsanity.com/paper/2306.09335/) (2023)
- [Distributional conformal prediction](https://mlsanity.com/paper/1909.07889/) (2019)
- [Large language model validity via enhanced conformal prediction methods](https://mlsanity.com/paper/2406.09714/) (2024)
- [Robust Validation: Confident Predictions Even When Distributions Shift](https://mlsanity.com/paper/2008.04267/) (2020)
- [Conformal Prediction using Conditional Histograms](https://mlsanity.com/paper/2105.08747/) (2021)
- [CD-split and HPD-split: efficient conformal regions in high dimensions](https://mlsanity.com/paper/2007.12778/) (2020)
- [Batch Multivalid Conformal Prediction](https://mlsanity.com/paper/2209.15145/) (2022)
- [Flexible distribution-free conditional predictive bands using density estimators](https://mlsanity.com/paper/1910.05575/) (2019)
- [PAC Confidence Sets for Deep Neural Networks via Calibrated Prediction](https://mlsanity.com/paper/2001.00106/) (2019)
- [Regression Conformal Prediction with Nearest Neighbours](https://mlsanity.com/paper/1401.3880/) (2014)
- [Improving Expert Predictions with Conformal Prediction](https://mlsanity.com/paper/2201.12006/) (2022)
- [Selection and Aggregation of Conformal Prediction Sets](https://mlsanity.com/paper/2104.13871/) (2021)

## Built on by (most cited first, 30 of 39)

- [Decision Theoretic Foundations for Conformal Prediction: Optimal Uncertainty Quantification for Risk-Averse Agents](https://mlsanity.com/paper/2502.02561/) (2025)
- [Minimum Volume Conformal Sets for Multivariate Regression](https://mlsanity.com/paper/2503.19068/) (2025)
- [Prune 'n Predict: Optimizing LLM Decision-making with Conformal Prediction](https://mlsanity.com/paper/2501.00555/) (2024)
- [Backward Conformal Prediction](https://mlsanity.com/paper/2505.13732/) (2025)
- [Conformal Inference under High-Dimensional Covariate Shifts via Likelihood-Ratio Regularization](https://mlsanity.com/paper/2502.13030/) (2025)
- [Conformal Risk Minimization with Variance Reduction](https://mlsanity.com/paper/2411.01696/) (2024)
- [Volume Optimality in Conformal Prediction with Structured Prediction Sets](https://mlsanity.com/paper/2502.16658/) (2025)
- [Conformal Risk Control for Semantic Uncertainty Quantification in Computed Tomography](https://mlsanity.com/paper/2503.00136/) (2025)
- [Direct Prediction Set Minimization via Bilevel Conformal Classifier Training](https://mlsanity.com/paper/2506.06599/) (2025)
- [Parametric Scaling Law of Tuning Bias in Conformal Prediction](https://mlsanity.com/paper/2502.03023/) (2025)
- [Conditional Conformal Risk Adaptation](https://mlsanity.com/paper/2504.07611/) (2025)
- [Conformal Information Pursuit for Interactively Guiding Large Language Models](https://mlsanity.com/paper/2507.03279/) (2025)
- [Conformal Prediction for Long-Tailed Classification](https://mlsanity.com/paper/2507.06867/) (2025)
- [Human-AI Collaborative Uncertainty Quantification](https://mlsanity.com/paper/2510.23476/) (2025)
- [Conformal Prediction Beyond the Seen: A Missing Mass Perspective for Uncertainty Quantification in Generative Models](https://mlsanity.com/paper/2506.05497/) (2025)
- [Domain-Shift-Aware Conformal Prediction for Large Language Models](https://mlsanity.com/paper/2510.05566/) (2025)
- [Non-exchangeable Conformal Prediction with Optimal Transport: Tackling Distribution Shifts with Unlabeled Data](https://mlsanity.com/paper/2507.10425/) (2025)
- [Conformal Prediction in The Loop: A Feedback-Based Uncertainty Model for Trajectory Optimization](https://mlsanity.com/paper/2510.16376/) (2025)
- [Singleton-Optimized Conformal Prediction](https://mlsanity.com/paper/2509.24095/) (2025)
- [Non-Asymptotic Analysis of Efficiency in Conformalized Regression](https://mlsanity.com/paper/2510.07093/) (2025)
- [CLAPS: Aleatoric-Epistemic Scaling via Last-Layer Laplace for Conformal Regression](https://mlsanity.com/paper/2512.01384/) (2025)
- [Co-optimization for Adaptive Conformal Prediction](https://mlsanity.com/paper/2603.01719/) (2026)
- [Conformal Risk-Averse Decision Making with Action Conditional Guarantee](https://mlsanity.com/paper/2606.05551/) (2026)
- [Dual-Splitting Conformal Prediction for Multi-Step Time Series Forecasting](https://mlsanity.com/paper/2503.21251/) (2025)
- [Learning Polyhedral Conformal Sets for Robust Optimization](https://mlsanity.com/paper/2605.08506/) (2026)
- [Empirical Bayes Conformal Prediction for Vision and Language Models](https://mlsanity.com/paper/2605.23189/) (2026)
- [On Optimal Data Splitting for Split Conformal Prediction](https://mlsanity.com/paper/2606.31600/) (2026)
- [Data-light Uncertainty Set Merging with Admissibility](https://mlsanity.com/paper/2410.12201/) (2024)
- [Feature Fitted Online Conformal Prediction for Deep Time Series Forecasting Model](https://mlsanity.com/paper/2505.08158/) (2025)
- [Smoothing-Based Conformal Prediction for Balancing Efficiency and Interpretability](https://mlsanity.com/paper/2509.22529/) (2025)

## Abstract (arXiv)

Conditional validity and length efficiency are two crucial aspects of conformal prediction (CP). Conditional validity ensures accurate uncertainty quantification for data subpopulations, while proper length efficiency ensures that the prediction sets remain informative. Despite significant efforts to address each of these issues individually, a principled framework that reconciles these two objectives has been missing in the CP literature. In this paper, we develop Conformal Prediction with Length-Optimization (CPL) - a novel and practical framework that constructs prediction sets with (near-) optimal length while ensuring conditional validity under various classes of covariate shifts, including the key cases of marginal and group-conditional coverage. In the infinite sample regime, we provide strong duality results which indicate that CPL achieves conditional validity and length optimality. In the finite sample regime, we show that CPL constructs conditionally valid prediction sets. Our extensive empirical evaluations demonstrate the superior prediction set size performance of CPL compared to state-of-the-art methods across diverse real-world and synthetic datasets in classification, regression, and large language model-based multiple choice question answering. An Implementation of our algorithm can be accessed at the following link: https://github.com/shayankiyani98/CP.