# MAPIE: an open-source library for distribution-free uncertainty quantification

arXiv: https://arxiv.org/abs/2207.12274 · HTML: https://mlsanity.com/paper/2207.12274/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 19 references that are in the map.
- First posted on arXiv 25 Jul 2022.

## Builds on (most cited first, 12 of 12)

- [Conformalized Quantile Regression](https://mlsanity.com/paper/1905.03222/) (2019)
- [Conformal Prediction Under Covariate Shift](https://mlsanity.com/paper/1904.06019/) (2019)
- [Classification with Valid and Adaptive Coverage](https://mlsanity.com/paper/2006.02544/) (2020)
- [Adaptive Conformal Inference Under Distribution Shift](https://mlsanity.com/paper/2106.00170/) (2021)
- [Uncertainty Sets for Image Classifiers using Conformal Prediction](https://mlsanity.com/paper/2009.14193/) (2020)
- [Distribution-Free, Risk-Controlling Prediction Sets](https://mlsanity.com/paper/2101.02703/) (2021)
- [Least Ambiguous Set-Valued Classifiers with Bounded Error Levels](https://mlsanity.com/paper/1609.00451/) (2016)
- [Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Control](https://mlsanity.com/paper/2110.01052/) (2021)
- [Predictive inference with the jackknife+](https://mlsanity.com/paper/1905.02928/) (2019)
- [Distribution-free uncertainty quantification for classification under label shift](https://mlsanity.com/paper/2103.03323/) (2021)
- [Predictive Inference Is Free with the Jackknife+-after-Bootstrap](https://mlsanity.com/paper/2002.09025/) (2020)
- [Copula-based conformal prediction for Multi-Target Regression](https://mlsanity.com/paper/2101.12002/) (2021)

## Built on by (most cited first, 17 of 17)

- [Analyzing Uncertainty of LLM-as-a-Judge: Interval Evaluations with Conformal Prediction](https://mlsanity.com/paper/2509.18658/) (2025)
- [Interpretable Machine Learning for TabPFN](https://mlsanity.com/paper/2403.10923/) (2024)
- [Uncertainty quantification for probabilistic machine learning in earth observation using conformal prediction](https://mlsanity.com/paper/2401.06421/) (2024)
- [Towards Fair In-Context Learning with Tabular Foundation Models](https://mlsanity.com/paper/2505.09503/) (2025)
- [Conformal time series decomposition with component-wise exchangeability](https://mlsanity.com/paper/2406.16766/) (2024)
- [On the Performance of LLMs for Real Estate Appraisal](https://mlsanity.com/paper/2506.11812/) (2025)
- [From Conformal Predictions to Confidence Regions](https://mlsanity.com/paper/2405.18601/) (2024)
- [Adaptive Conformal Inference for Multi-Step Ahead Time-Series Forecasting Online](https://mlsanity.com/paper/2409.14792/) (2024)
- [From Group-Differences to Single-Subject Probability: Conformal Prediction-based Uncertainty Estimation for Brain-Age Modeling](https://mlsanity.com/paper/2302.05304/) (2023)
- [Reliable Statistical Guarantees for Conformal Predictors with Small Datasets](https://mlsanity.com/paper/2512.04566/) (2025)
- [Machine Learning Optimized Approach for Parameter Selection in MESHFREE Simulations](https://mlsanity.com/paper/2403.13672/) (2024)
- [Onboard Out-of-Calibration Detection of Deep Learning Models using Conformal Prediction](https://mlsanity.com/paper/2405.02634/) (2024)
- [Enhancing Deployment-Time Predictive Model Robustness for Code Analysis and Optimization](https://mlsanity.com/paper/2501.00298/) (2024)
- [Flood Risk Follows Valleys, Not Grids: Graph Neural Networks for Flash Flood Susceptibility Mapping in Himachal Pradesh with Conformal Uncertainty Quantification](https://mlsanity.com/paper/2603.15681/) (2026)
- [Accurate and Reliable Uncertainty Estimates for Deterministic Predictions Extensions to Under and Overpredictions](https://mlsanity.com/paper/2604.08755/) (2026)
- [Calibration, Uncertainty Communication, and Deployment Readiness in CKD Risk Prediction: A Framework Evaluation Study](https://mlsanity.com/paper/2605.21566/) (2026)
- [Extreme Binary Classification: Extreme Value Theory for Extreme Constraint on False Negative](https://mlsanity.com/paper/2610.09984/) (2026)

## Abstract (arXiv)

Estimating uncertainties associated with the predictions of Machine Learning (ML) models is of crucial importance to assess their robustness and predictive power. In this submission, we introduce MAPIE (Model Agnostic Prediction Interval Estimator), an open-source Python library that quantifies the uncertainties of ML models for single-output regression and multi-class classification tasks. MAPIE implements conformal prediction methods, allowing the user to easily compute uncertainties with strong theoretical guarantees on the marginal coverages and with mild assumptions on the model or on the underlying data distribution. MAPIE is hosted on scikit-learn-contrib and is fully "scikit-learn-compatible". As such, it accepts any type of regressor or classifier coming with a scikit-learn API. The library is available at: https://github.com/scikit-learn-contrib/MAPIE/.