mlsanityData to 14 Oct 2026

Conformal Off-policy Prediction

arXiv 2206.06711 ↗First posted 14 Jun 2022stat.ML cs.LGcited by 14

Not on the map. Why?

Abstract (arXiv)

Off-policy evaluation is critical in a number of applications where new policies need to be evaluated offline before online deployment. Most existing methods focus on the expected return, define the target parameter through averaging and provide a point estimator only. In this paper, we develop a novel procedure to produce reliable interval estimators for a target policy's return starting from any initial state. Our proposal accounts for the variability of the return around its expectation, focuses on the individual effect and offers valid uncertainty quantification. Our main idea lies in designing a pseudo policy that generates subsamples as if they were sampled from the target policy so that existing conformal prediction algorithms are applicable to prediction interval construction. Our methods are justified by theories, synthetic data and real data from short-video platforms.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 39 of its 65 references that are in the map. First posted on arXiv 14 Jun 2022.

Builds on

39 of 65 references.

References · 39

  1. 2017A Distributional Perspective on Reinforcement Learningcited by 765
  2. 2016Distribution-Free Predictive Inference For Regressioncited by 668
  3. 2019Conformalized Quantile Regressioncited by 629
  4. 2019Conformal Prediction Under Covariate Shiftcited by 519
  5. 2017Distributional Reinforcement Learning with Quantile Regressioncited by 474
  6. 2015Doubly Robust Off-policy Value Evaluation for Reinforcement Learningcited by 339
  7. 2019DualDICE: Behavior-Agnostic Estimation of Discounted Stationary Distribution Correctionscited by 254
  8. 2019Batch Policy Learning under Constraintscited by 244
  9. 2018Breaking the Curse of Horizon: Infinite-Horizon Off-Policy Estimationcited by 227
  10. 2020Conformal Inference of Counterfactuals and Individual Treatment Effectscited by 151
  11. 2019Minimax Weight and Q-Function Learning for Off-Policy Evaluationcited by 150
  12. 2018More Robust Doubly Robust Off-policy Evaluationcited by 141
27 more
  1. 2019Double Reinforcement Learning for Efficient Off-Policy Evaluation in Markov Decision Processescited by 119
  2. 2021Distribution-free uncertainty quantification for classification under label shiftcited by 101
  3. 2019Nested conformal prediction and quantile out-of-bag ensemble methodscited by 89
  4. 2013A Conformal Prediction Approach to Explore Functional Datacited by 86
  5. 2019A comparison of some conformal quantile regression methodscited by 76
  6. 2020Batch Policy Learning in Average Reward Markov Decision Processescited by 69
  7. 2016Estimating Dynamic Treatment Regimes in Mobile Health Using V-learningcited by 65
  8. 2020Statistical Inference of the Value Function for Reinforcement Learning in Infinite Horizon Settingscited by 65
  9. 2020CoinDICE: Off-Policy Confidence Interval Estimationcited by 64
  10. 2021Sensitivity Analysis of Individual Treatment Effects: A Robust Conformal Inference Approachcited by 62
  11. 2019Distributional Reinforcement Learning for Efficient Explorationcited by 60
  12. 2018Policy Evaluation and Optimization with Continuous Treatmentscited by 58
  13. 2021Conformalized Survival Analysiscited by 53
  14. 2019Doubly Robust Bias Reduction in Infinite Horizon Off-Policy Estimationcited by 48
  15. 2021Universal Off-Policy Evaluationcited by 43
  16. 2019Off-Policy Estimation of Long-Term Average Outcomes with Applications to Mobile Healthcited by 42
  17. 2021Bootstrapping Fitted Q-Evaluation for Off-Policy Inferencecited by 40
  18. 2021Conformal Sensitivity Analysis for Individual Treatment Effectscited by 40
  19. 2020Accountable Off-Policy Evaluation With Kernel Bellman Statisticscited by 36
  20. 2021Deeply-Debiased Off-Policy Interval Estimationcited by 36
  21. 2022On Well-posedness and Minimax Optimal Rates of Nonparametric Q-function Estimation in Off-policy Evaluationcited by 33
  22. 2019Importance Resampling for Off-policy Predictioncited by 26
  23. 2022Conformal Off-Policy Prediction in Contextual Banditscited by 25
  24. 2021Multi Split Conformal Predictioncited by 21
  25. 2021Projected State-action Balancing Weights for Offline Reinforcement Learningcited by 16
  26. 2020Conformal prediction intervals for the individual treatment effectcited by 10
  27. 2022Policy Learning for Optimal Individualized Dose Intervalscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 3; 2025: 4; 2026: 5.

14 papers.

Sort:

By area:

  1. 2023Conformal Meta-learners for Predictive Inference of Individual Treatment Effectscited by 23
  2. 2024Conformal Prediction for Causal Effects of Continuous Treatmentscited by 19
  3. 2026Conformal Policy Controlcited by 6
  4. 2023When is Off-Policy Evaluation (Reward Modeling) Useful in Contextual Bandits? A Data-Centric Perspectivecited by 4
  5. 2024Conformal Off-Policy Prediction for Multi-Agent Systemscited by 3
  6. 2024Efficient Online Set-valued Classification with Bandit Feedbackcited by 3
  7. 2025Bellman Calibration for V-Learning in Offline Reinforcement Learningcited by 3
  8. 2025Doubly Robust Fusion of Many Treatments for Policy Learningcited by 2
  9. 2025Conformal Prediction Beyond the Horizon: Distribution-Free Inference for Policy Evaluationcited by 2
  10. 2025A Systematic Review of Conformal Inference Procedures for Treatment Effect Estimation: Methods and Challengescited by 1
  11. 2026Compact Conformal Subgraphscited by 0
  12. 2026Robust Sequential Experimental Design for A/B Testingcited by 0
2 more
  1. 2026Set-Valued Policy Learningcited by 0
  2. 2026Counterfactual Online Conformal Prediction Under Adaptive Loggingcited by 0