# Selection and Aggregation of Conformal Prediction Sets

arXiv: https://arxiv.org/abs/2104.13871 · HTML: https://mlsanity.com/paper/2104.13871/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Apr 2021.

## Built on by (most cited first, 30 of 49)

- [Improved Online Conformal Prediction via Strongly Adaptive Online Learning](https://mlsanity.com/paper/2302.07869/) (2023)
- [Training Uncertainty-Aware Classifiers with Conformalized Deep Learning](https://mlsanity.com/paper/2205.05878/) (2022)
- [Length Optimization in Conformal Prediction](https://mlsanity.com/paper/2406.18814/) (2024)
- [Predictive Inference with Feature Conformal Prediction](https://mlsanity.com/paper/2210.00173/) (2022)
- [Minimum Volume Conformal Sets for Multivariate Regression](https://mlsanity.com/paper/2503.19068/) (2025)
- [Efficient and Differentiable Conformal Prediction with General Function Classes](https://mlsanity.com/paper/2202.11091/) (2022)
- [Prune 'n Predict: Optimizing LLM Decision-making with Conformal Prediction](https://mlsanity.com/paper/2501.00555/) (2024)
- [Conformal inference is (almost) free for neural networks trained with early stopping](https://mlsanity.com/paper/2301.11556/) (2023)
- [Conformal Methods for Quantifying Uncertainty in Spatiotemporal Data: A Survey](https://mlsanity.com/paper/2209.03580/) (2022)
- [Conformal online model aggregation](https://mlsanity.com/paper/2403.15527/) (2024)
- [Weighted Aggregation of Conformity Scores for Classification](https://mlsanity.com/paper/2407.10230/) (2024)
- [Conformal Risk Minimization with Variance Reduction](https://mlsanity.com/paper/2411.01696/) (2024)
- [Debate as Optimization: Adaptive Conformal Prediction and Diverse Retrieval for Event Extraction](https://mlsanity.com/paper/2406.12197/) (2024)
- [COLEP: Certifiably Robust Learning-Reasoning Conformal Prediction via Probabilistic Circuits](https://mlsanity.com/paper/2403.11348/) (2024)
- [Conformal Prediction for Ensembles: Improving Efficiency via Score-Based Aggregation](https://mlsanity.com/paper/2405.16246/) (2024)
- [Entropy Reweighted Conformal Classification](https://mlsanity.com/paper/2407.17377/) (2024)
- [Parametric Scaling Law of Tuning Bias in Conformal Prediction](https://mlsanity.com/paper/2502.03023/) (2025)
- [PAC-Bayes Generalization Certificates for Learned Inductive Conformal Prediction](https://mlsanity.com/paper/2312.04658/) (2023)
- [Volume-Sorted Prediction Set: Efficient Conformal Prediction for Multi-Target Regression](https://mlsanity.com/paper/2503.02205/) (2025)
- [Improving the statistical efficiency of cross-conformal prediction](https://mlsanity.com/paper/2503.01495/) (2025)
- [Valid Selection among Conformal Sets](https://mlsanity.com/paper/2506.20173/) (2025)
- [On the Temperature of Bayesian Graph Neural Networks for Conformal Prediction](https://mlsanity.com/paper/2310.11479/) (2023)
- [Multi-model Online Conformal Prediction with Graph-Structured Feedback](https://mlsanity.com/paper/2506.20898/) (2025)
- [A note on conditional PAC-efficient reasoning in large language model routing](https://mlsanity.com/paper/2512.03057/) (2025)
- [Tightening optimality gap with confidence through conformal prediction](https://mlsanity.com/paper/2503.04071/) (2025)
- [Probabilistic Conformal Coverage Guarantees in Small-Data Settings](https://mlsanity.com/paper/2509.15349/) (2025)
- [Singleton-Optimized Conformal Prediction](https://mlsanity.com/paper/2509.24095/) (2025)
- [Uncertainty-Guided Model Selection for Tabular Foundation Models in Biomolecule Efficacy Prediction](https://mlsanity.com/paper/2510.02476/) (2025)
- [Conformal Prediction Assessment: A Framework for Conditional Coverage Evaluation and Selection](https://mlsanity.com/paper/2603.27189/) (2026)
- [How Nonconformity Functions and Difficulty of Datasets Impact the Efficiency of Conformal Classifiers](https://mlsanity.com/paper/2108.05677/) (2021)

## Abstract (arXiv)

Conformal prediction is a generic methodology for finite-sample valid distribution-free prediction. This technique has garnered a lot of attention in the literature partly because it can be applied with any machine learning algorithm that provides point predictions to yield valid prediction regions. Of course, the efficiency (width/volume) of the resulting prediction region depends on the performance of the machine learning algorithm. In the context of point prediction, several techniques (such as cross-validation) exist to select one of many machine learning algorithms for better performance. In contrast, such selection techniques are seldom discussed in the context of set prediction (or prediction regions). In this paper, we consider the problem of obtaining the smallest conformal prediction region given a family of machine learning algorithms. We provide two general-purpose selection algorithms and consider coverage as well as width properties of the final prediction region. The first selection method yields the smallest width prediction region among the family of conformal prediction regions for all sample sizes but only has an approximate coverage guarantee. The second selection method has a finite sample coverage guarantee but only attains close to the smallest width. The approximate optimal width property of the second method is quantified via an oracle inequality. As an illustration, we consider the use of aggregation of non-parametric regression estimators in the split conformal method with the absolute residual conformal score.