# Generalized Venn and Venn-Abers Calibration with Applications in Conformal Prediction

arXiv: https://arxiv.org/abs/2502.05676 · HTML: https://mlsanity.com/paper/2502.05676/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 59 references that are in the map.
- First posted on arXiv 8 Feb 2025.

## Builds on (most cited first, 22 of 22)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Distribution-Free Predictive Inference For Regression](https://mlsanity.com/paper/1604.04173/) (2016)
- [Conformalized Quantile Regression](https://mlsanity.com/paper/1905.03222/) (2019)
- [Multiaccuracy: Black-Box Post-Processing for Fairness in Classification](https://mlsanity.com/paper/1805.12317/) (2018)
- [Conformal Prediction With Conditional Guarantees](https://mlsanity.com/paper/2305.12616/) (2023)
- [Distribution-free binary classification: prediction sets, confidence intervals and calibration](https://mlsanity.com/paper/2006.10564/) (2020)
- [Classifier Calibration: A survey on how to assess and improve predicted class probabilities](https://mlsanity.com/paper/2112.10327/) (2021)
- [Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making](https://mlsanity.com/paper/1802.01029/) (2018)
- [Batch Multivalid Conformal Prediction](https://mlsanity.com/paper/2209.15145/) (2022)
- [Moment Multicalibration for Uncertainty Estimation](https://mlsanity.com/paper/2008.08037/) (2020)
- [Distribution-free calibration guarantees for histogram binning without sample splitting](https://mlsanity.com/paper/2105.04656/) (2021)
- [A Survey on Intersectional Fairness in Machine Learning: Notions, Mitigation, and Challenges](https://mlsanity.com/paper/2305.06969/) (2023)
- [Regression Diagnostics meets Forecast Evaluation: Conditional Calibration, Reliability Diagrams, and Coefficient of Determination](https://mlsanity.com/paper/2108.03210/) (2021)
- [A Unifying Perspective on Multi-Calibration: Game Dynamics for Multi-Objective Learning](https://mlsanity.com/paper/2302.10863/) (2023)
- [Causal isotonic calibration for heterogeneous treatment effects](https://mlsanity.com/paper/2302.14011/) (2023)
- [Self-Calibrating Conformal Prediction](https://mlsanity.com/paper/2402.07307/) (2024)
- [Stabilized Inverse Probability Weighting via Isotonic Calibration](https://mlsanity.com/paper/2411.06342/) (2024)
- [The Scope of Multicalibration: Characterizing Multicalibration via Property Elicitation](https://mlsanity.com/paper/2302.08507/) (2023)
- [Orthogonal Causal Calibration](https://mlsanity.com/paper/2406.01933/) (2024)
- [Forster-Warmuth Counterfactual Regression: A Unified Learning Approach](https://mlsanity.com/paper/2307.16798/) (2023)
- [Well-Calibrated Probabilistic Predictive Maintenance using Venn-Abers](https://mlsanity.com/paper/2306.06642/) (2023)

## Built on by (most cited first, 7 of 7)

- [Calibrate, Don't Curate: Label-Efficient Estimation from Noisy LLM Judges](https://mlsanity.com/paper/2605.09702/) (2026)
- [Calibeating Prediction-Powered Inference](https://mlsanity.com/paper/2604.21260/) (2026)
- [Bellman Calibration for V-Learning in Offline Reinforcement Learning](https://mlsanity.com/paper/2512.23694/) (2025)
- [Learning density ratios in causal inference using Bregman-Riesz regression](https://mlsanity.com/paper/2510.16127/) (2025)
- [Residual Distribution Predictive Systems](https://mlsanity.com/paper/2510.26914/) (2025)
- [Inductive Venn-Abers and related regressors](https://mlsanity.com/paper/2605.06646/) (2026)
- [Bellman Calibration for Marginalized Importance Weighting in Offline Reinforcement Learning](https://mlsanity.com/paper/2608.24858/) (2026)

## Abstract (arXiv)

Ensuring model calibration is critical for reliable prediction, yet popular distribution-free methods such as histogram binning and isotonic regression offer only asymptotic guarantees. We introduce a unified framework for Venn and Venn-Abers calibration that extends Vovk's approach beyond binary classification to a broad class of prediction problems defined by generic loss functions. Our method transforms any perfectly in-sample calibrated predictor into a set-valued predictor that, in finite samples, outputs at least one marginally calibrated point prediction. These set predictions shrink asymptotically and converge to a single conditionally calibrated prediction, capturing epistemic uncertainty. We further propose Venn multicalibration, a new approach for achieving finite-sample calibration across subpopulations. For quantile loss, our framework recovers group-conditional and multicalibrated conformal prediction as special cases and yields novel prediction intervals with quantile-conditional coverage.