# Conformal Off-Policy Prediction in Contextual Bandits

arXiv: https://arxiv.org/abs/2206.04405 · HTML: https://mlsanity.com/paper/2206.04405/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 39 references that are in the map.
- First posted on arXiv 9 Jun 2022.

## Builds on (most cited first, 22 of 22)

- [A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification](https://mlsanity.com/paper/2107.07511/) (2021)
- [A Distributional Perspective on Reinforcement Learning](https://mlsanity.com/paper/1707.06887/) (2017)
- [Conformalized Quantile Regression](https://mlsanity.com/paper/1905.03222/) (2019)
- [Conformal Prediction Under Covariate Shift](https://mlsanity.com/paper/1904.06019/) (2019)
- [Doubly Robust Policy Evaluation and Optimization](https://mlsanity.com/paper/1503.02834/) (2015)
- [Conformal Inference of Counterfactuals and Individual Treatment Effects](https://mlsanity.com/paper/2006.06138/) (2020)
- [More Robust Doubly Robust Off-policy Evaluation](https://mlsanity.com/paper/1802.03493/) (2018)
- [Introducing LETOR 4.0 Datasets](https://mlsanity.com/paper/1306.2597/) (2013)
- [Towards Optimal Off-Policy Evaluation for Reinforcement Learning with Marginalized Importance Sampling](https://mlsanity.com/paper/1906.03393/) (2019)
- [Learning Optimal Conformal Classifiers](https://mlsanity.com/paper/2110.09192/) (2021)
- [Optimal and Adaptive Off-policy Evaluation in Contextual Bandits](https://mlsanity.com/paper/1612.01205/) (2016)
- [Off-policy evaluation for slate recommendation](https://mlsanity.com/paper/1605.04812/) (2016)
- [Doubly robust off-policy evaluation with shrinkage](https://mlsanity.com/paper/1907.09623/) (2019)
- [A Contextual Bandit Bake-off](https://mlsanity.com/paper/1802.04064/) (2018)
- [Being Optimistic to Be Conservative: Quickly Learning a CVaR Policy](https://mlsanity.com/paper/1911.01546/) (2019)
- [Sensitivity Analysis of Individual Treatment Effects: A Robust Conformal Inference Approach](https://mlsanity.com/paper/2111.12161/) (2021)
- [Universal Off-Policy Evaluation](https://mlsanity.com/paper/2104.12820/) (2021)
- [Conformal Sensitivity Analysis for Individual Treatment Effects](https://mlsanity.com/paper/2112.03493/) (2021)
- [Confident Off-Policy Evaluation and Selection through Self-Normalized Importance Weighting](https://mlsanity.com/paper/2006.10460/) (2020)
- [Off-Policy Risk Assessment in Contextual Bandits](https://mlsanity.com/paper/2104.08977/) (2021)
- [Best Arm Identification for Contaminated Bandits](https://mlsanity.com/paper/1802.09514/) (2018)
- [Learning Robust Decision Policies from Observational Data](https://mlsanity.com/paper/2006.02355/) (2020)

## Built on by (most cited first, 25 of 25)

- [Conformal Decision Theory: Safe Autonomous Decisions from Imperfect Predictions](https://mlsanity.com/paper/2310.05921/) (2023)
- [Probabilistic Conformal Prediction Using Conditional Random Samples](https://mlsanity.com/paper/2206.06584/) (2022)
- [Conformal Meta-learners for Predictive Inference of Individual Treatment Effects](https://mlsanity.com/paper/2308.14895/) (2023)
- [Conformal Prediction for Causal Effects of Continuous Treatments](https://mlsanity.com/paper/2407.03094/) (2024)
- [Conformal Off-policy Prediction](https://mlsanity.com/paper/2206.06711/) (2022)
- [Conformalized Interactive Imitation Learning: Handling Expert Shift and Intermittent Feedback](https://mlsanity.com/paper/2410.08852/) (2024)
- [Conformal Counterfactual Inference under Hidden Confounding](https://mlsanity.com/paper/2405.12387/) (2024)
- [Conformal Off-Policy Evaluation in Markov Decision Processes](https://mlsanity.com/paper/2304.02574/) (2023)
- [Conformalized Teleoperation: Confidently Mapping Human Inputs to High-Dimensional Robot Actions](https://mlsanity.com/paper/2406.07767/) (2024)
- [What If We Had Used a Different App? Reliable Counterfactual KPI Analysis in Wireless Systems](https://mlsanity.com/paper/2410.00150/) (2024)
- [Conformal Policy Control](https://mlsanity.com/paper/2603.02196/) (2026)
- [Distribution-Free Finite-Sample Guarantees and Split Conformal Prediction](https://mlsanity.com/paper/2210.14735/) (2022)
- [When is Off-Policy Evaluation (Reward Modeling) Useful in Contextual Bandits? A Data-Centric Perspective](https://mlsanity.com/paper/2311.14110/) (2023)
- [PERRY: Policy Evaluation with Confidence Intervals using Auxiliary Data](https://mlsanity.com/paper/2507.20068/) (2025)
- [Off-Policy Evaluation with Out-of-Sample Guarantees](https://mlsanity.com/paper/2301.08649/) (2023)
- [Conformal Off-Policy Prediction for Multi-Agent Systems](https://mlsanity.com/paper/2403.16871/) (2024)
- [Efficient Online Set-valued Classification with Bandit Feedback](https://mlsanity.com/paper/2405.04393/) (2024)
- [Doubly Robust Fusion of Many Treatments for Policy Learning](https://mlsanity.com/paper/2505.08092/) (2025)
- [Bayesian Counterfactual Mean Embeddings and Off-Policy Evaluation](https://mlsanity.com/paper/2211.01518/) (2022)
- [AdaptNC: Adaptive Nonconformity Scores for Conformal Prediction under Distribution Shift](https://mlsanity.com/paper/2602.01629/) (2026)
- [Set-Valued Policy Learning](https://mlsanity.com/paper/2605.19830/) (2026)
- [Safety Screening for Voltage Control in Active Distribution Grids via Distributionally Robust Conformal Screening](https://mlsanity.com/paper/2608.30889/) (2026)
- [Certified Safety Curation: Distribution-Free Guarantees for Safe Offline Reinforcement Learning](https://mlsanity.com/paper/2609.12014/) (2026)
- [Model-based Bootstrap for Offline Policy Evaluation in Tabular Reinforcement Learning](https://mlsanity.com/paper/2609.20389/) (2026)
- [Counterfactual Online Conformal Prediction Under Adaptive Logging](https://mlsanity.com/paper/2609.30811/) (2026)

## Abstract (arXiv)

Most off-policy evaluation methods for contextual bandits have focused on the expected outcome of a policy, which is estimated via methods that at best provide only asymptotic guarantees. However, in many applications, the expectation may not be the best measure of performance as it does not capture the variability of the outcome. In addition, particularly in safety-critical settings, stronger guarantees than asymptotic correctness may be required. To address these limitations, we consider a novel application of conformal prediction to contextual bandits. Given data collected under a behavioral policy, we propose conformal off-policy prediction (COPP), which can output reliable predictive intervals for the outcome under a new target policy. We provide theoretical finite-sample guarantees without making any additional assumptions beyond the standard contextual bandit setup, and empirically demonstrate the utility of COPP compared with existing methods on synthetic and real-world data.