# Large-scale probabilistic predictors with and without guarantees of validity

arXiv: https://arxiv.org/abs/1511.00213 · HTML: https://mlsanity.com/paper/1511.00213/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Nov 2015.

## Built on by (most cited first, 18 of 18)

- [Conformal Language Modeling](https://mlsanity.com/paper/2306.10193/) (2023)
- [Conformal Prediction Sets with Limited False Positives](https://mlsanity.com/paper/2202.07650/) (2022)
- [Efficiently Controlling Multiple Risks with Pareto Testing](https://mlsanity.com/paper/2210.07913/) (2022)
- [Rethinking Early Stopping: Refine, Then Calibrate](https://mlsanity.com/paper/2501.19195/) (2025)
- [Calibrated Explanations: with Uncertainty Information and Counterfactuals](https://mlsanity.com/paper/2305.02305/) (2023)
- [Calibration of Natural Language Understanding Models with Venn--ABERS Predictors](https://mlsanity.com/paper/2205.10586/) (2022)
- [Risk-Controlling Model Selection via Guided Bayesian Optimization](https://mlsanity.com/paper/2312.01692/) (2023)
- [Calibrated Large Language Models for Binary Question Answering](https://mlsanity.com/paper/2407.01122/) (2024)
- [Conformal Predictive Distributions for Order Fulfillment Time Forecasting](https://mlsanity.com/paper/2505.17340/) (2025)
- [CalArena: A Large-Scale Post-Hoc Calibration Benchmark](https://mlsanity.com/paper/2605.30188/) (2026)
- [Exploring the Impact of Explainable AI and Cognitive Capabilities on Users' Decisions](https://mlsanity.com/paper/2505.01192/) (2025)
- [Adaptive Cumulative Mass Calibration with Conformal Prediction](https://mlsanity.com/paper/2505.15437/) (2025)
- [A systematic evaluation of uncertainty quantification techniques in deep learning: a case study in photoplethysmography signal analysis](https://mlsanity.com/paper/2511.00301/) (2025)
- [Approximate Conditional Coverage & Calibration via Neural Model Approximations](https://mlsanity.com/paper/2205.14310/) (2022)
- [Out-of-Sample Hydrocarbon Production Forecasting: Time Series Machine Learning using Productivity Index-Driven Features and Inductive Conformal Prediction](https://mlsanity.com/paper/2508.14078/) (2025)
- [Protected Probabilistic Classification Library](https://mlsanity.com/paper/2509.11267/) (2025)
- [Residual Distribution Predictive Systems](https://mlsanity.com/paper/2510.26914/) (2025)
- [Inductive Venn-Abers and related regressors](https://mlsanity.com/paper/2605.06646/) (2026)

## Abstract (arXiv)

This paper studies theoretically and empirically a method of turning machine-learning algorithms into probabilistic predictors that automatically enjoys a property of validity (perfect calibration) and is computationally efficient. The price to pay for perfect calibration is that these probabilistic predictors produce imprecise (in practice, almost precise for large data sets) probabilities. When these imprecise probabilities are merged into precise probabilities, the resulting predictors, while losing the theoretical property of perfect calibration, are consistently more accurate than the existing methods in empirical studies.