# Generalization and Informativeness of Conformal Prediction

arXiv: https://arxiv.org/abs/2401.11810 · HTML: https://mlsanity.com/paper/2401.11810/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 22 references that are in the map.
- First posted on arXiv 22 Jan 2024.

## Builds on (most cited first, 10 of 10)

- [Least Ambiguous Set-Valued Classifiers with Bounded Error Levels](https://mlsanity.com/paper/1609.00451/) (2016)
- [User-friendly introduction to PAC-Bayes bounds](https://mlsanity.com/paper/2110.11216/) (2021)
- [Learning Optimal Conformal Classifiers](https://mlsanity.com/paper/2110.09192/) (2021)
- [Generalization Error Bounds for Noisy, Iterative Algorithms](https://mlsanity.com/paper/1801.04295/) (2018)
- [Generalization Bounds: Perspectives from Information Theory and PAC-Bayes](https://mlsanity.com/paper/2309.04381/) (2023)
- [Calibrating Noise to Variance in Adaptive Data Analysis](https://mlsanity.com/paper/1712.07196/) (2017)
- [Forking Uncertainties: Reliable Prediction and Model Predictive Control with Sequence Models via Conformal Risk Control](https://mlsanity.com/paper/2310.10299/) (2023)
- [Few-Shot Calibration of Set Predictors via Meta-Learned Cross-Validation-Based Conformal Prediction](https://mlsanity.com/paper/2210.03067/) (2022)
- [On the Expected Size of Conformal Prediction Sets](https://mlsanity.com/paper/2306.07254/) (2023)
- [PAC-Bayes Generalization Certificates for Learned Inductive Conformal Prediction](https://mlsanity.com/paper/2312.04658/) (2023)

## Built on by (most cited first, 9 of 9)

- [Length Optimization in Conformal Prediction](https://mlsanity.com/paper/2406.18814/) (2024)
- [Conformal Calibration: Ensuring the Reliability of Black-Box AI in Wireless Systems](https://mlsanity.com/paper/2504.09310/) (2025)
- [On the Impact of Uncertainty and Calibration on Likelihood-Ratio Membership Inference Attacks](https://mlsanity.com/paper/2402.10686/) (2024)
- [Distributed Conformal Prediction via Message Passing](https://mlsanity.com/paper/2501.14544/) (2025)
- [Dynamic Estimation Loss Control in Variational Quantum Sensing via Online Conformal Inference](https://mlsanity.com/paper/2505.23389/) (2025)
- [Generalization and Informativeness of Weighted Conformal Risk Control Under Covariate Shift](https://mlsanity.com/paper/2501.11413/) (2025)
- [Non-Asymptotic Analysis of Efficiency in Conformalized Regression](https://mlsanity.com/paper/2510.07093/) (2025)
- [Uncertainty Quantification and Data Efficiency in AI: An Information-Theoretic Perspective](https://mlsanity.com/paper/2512.05267/) (2025)
- [Conformal Changepoint Localization and Root Cause Analysis with Corrupted Observations](https://mlsanity.com/paper/2607.26481/) (2026)

## Abstract (arXiv)

The safe integration of machine learning modules in decision-making processes hinges on their ability to quantify uncertainty. A popular technique to achieve this goal is conformal prediction (CP), which transforms an arbitrary base predictor into a set predictor with coverage guarantees. While CP certifies the predicted set to contain the target quantity with a user-defined tolerance, it does not provide control over the average size of the predicted sets, i.e., over the informativeness of the prediction. In this work, a theoretical connection is established between the generalization properties of the base predictor and the informativeness of the resulting CP prediction sets. To this end, an upper bound is derived on the expected size of the CP set predictor that builds on generalization error bounds for the base predictor. The derived upper bound provides insights into the dependence of the average size of the CP set predictor on the amount of calibration data, the target reliability, and the generalization performance of the base predictor. The theoretical insights are validated using simple numerical regression and classification tasks.