# Predictive Inference Is Free with the Jackknife+-after-Bootstrap

arXiv: https://arxiv.org/abs/2002.09025 · HTML: https://mlsanity.com/paper/2002.09025/

## Key facts

- Cited by 55 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 20 Feb 2020.

## Built on by (most cited first, 30 of 55)

- [Testing for Outliers with Conformal p-values](https://mlsanity.com/paper/2104.08279/) (2021)
- [Conformal Prediction using Conditional Histograms](https://mlsanity.com/paper/2105.08747/) (2021)
- [Conformal prediction for multi-dimensional time series by ellipsoidal sets](https://mlsanity.com/paper/2403.03850/) (2024)
- [Conformal prediction under ambiguous ground truth](https://mlsanity.com/paper/2307.09302/) (2023)
- [Copula Conformal Prediction for Multi-step Time Series Forecasting](https://mlsanity.com/paper/2212.03281/) (2022)
- [Ensemble Conformalized Quantile Regression for Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2202.08756/) (2022)
- [Split Localized Conformal Prediction](https://mlsanity.com/paper/2206.13092/) (2022)
- [Integrative conformal p-values for powerful out-of-distribution testing with labeled outliers](https://mlsanity.com/paper/2208.11111/) (2022)
- [MAPIE: an open-source library for distribution-free uncertainty quantification](https://mlsanity.com/paper/2207.12274/) (2022)
- [Integrating Uncertainty Awareness into Conformalized Quantile Regression](https://mlsanity.com/paper/2306.08693/) (2023)
- [Learning Prediction Intervals for Regression: Generalization and Calibration](https://mlsanity.com/paper/2102.13625/) (2021)
- [Derandomized Novelty Detection with FDR Control via Conformal E-values](https://mlsanity.com/paper/2302.07294/) (2023)
- [Conformal inference is (almost) free for neural networks trained with early stopping](https://mlsanity.com/paper/2301.11556/) (2023)
- [Conformal Anomaly Detection on Spatio-Temporal Observations with Missing Data](https://mlsanity.com/paper/2105.11886/) (2021)
- [Looks Too Good To Be True: An Information-Theoretic Analysis of Hallucinations in Generative Restoration Models](https://mlsanity.com/paper/2405.16475/) (2024)
- [Kernel-based Optimally Weighted Conformal Time-Series Prediction](https://mlsanity.com/paper/2405.16828/) (2024)
- [Conformal Methods for Quantifying Uncertainty in Spatiotemporal Data: A Survey](https://mlsanity.com/paper/2209.03580/) (2022)
- [Regression Trees for Fast and Adaptive Prediction Intervals](https://mlsanity.com/paper/2402.07357/) (2024)
- [Weighted Aggregation of Conformity Scores for Classification](https://mlsanity.com/paper/2407.10230/) (2024)
- [PCS-UQ: Uncertainty Quantification via the Predictability-Computability-Stability Framework](https://mlsanity.com/paper/2505.08784/) (2025)
- [Full Conformal Adaptation of Medical Vision-Language Models](https://mlsanity.com/paper/2506.06076/) (2025)
- [Root-finding Approaches for Computing Conformal Prediction Set](https://mlsanity.com/paper/2104.06648/) (2021)
- [Conformal Credal Self-Supervised Learning](https://mlsanity.com/paper/2205.15239/) (2022)
- [Conformal Risk Control for Ordinal Classification](https://mlsanity.com/paper/2405.00417/) (2024)
- [Conformal prediction set for time-series](https://mlsanity.com/paper/2206.07851/) (2022)
- [What's Behind the Mask: Estimating Uncertainty in Image-to-Image Problems](https://mlsanity.com/paper/2211.15211/) (2022)
- [Explanation Shift: How Did the Distribution Shift Impact the Model?](https://mlsanity.com/paper/2303.08081/) (2023)
- [Improving the statistical efficiency of cross-conformal prediction](https://mlsanity.com/paper/2503.01495/) (2025)
- [Extreme Conformal Prediction: Reliable Intervals for High-Impact Events](https://mlsanity.com/paper/2505.08578/) (2025)
- [A Survey on Uncertainty Toolkits for Deep Learning](https://mlsanity.com/paper/2205.01040/) (2022)

## Abstract (arXiv)

Ensemble learning is widely used in applications to make predictions in complex decision problems---for example, averaging models fitted to a sequence of samples bootstrapped from the available training data. While such methods offer more accurate, stable, and robust predictions and model estimates, much less is known about how to perform valid, assumption-lean inference on the output of these types of procedures. In this paper, we propose the jackknife+-after-bootstrap (J+aB), a procedure for constructing a predictive interval, which uses only the available bootstrapped samples and their corresponding fitted models, and is therefore "free" in terms of the cost of model fitting. The J+aB offers a predictive coverage guarantee that holds with no assumptions on the distribution of the data, the nature of the fitted model, or the way in which the ensemble of models are aggregated---at worst, the failure rate of the predictive interval is inflated by a factor of 2. Our numerical experiments verify the coverage and accuracy of the resulting predictive intervals on real data.