Cross-Validated Off-Policy Evaluation
Not on the map. Why?
Abstract (arXiv)
We study estimator selection and hyper-parameter tuning in off-policy evaluation. Although cross-validation is the most popular method for model selection in supervised learning, off-policy evaluation relies mostly on theory, which provides only limited guidance to practitioners. We show how to use cross-validation for off-policy evaluation. This challenges a popular belief that cross-validation in off-policy evaluation is not feasible. We evaluate our method empirically and show that it addresses a variety of use cases.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 20 of its 46 references that are in the map. First posted on arXiv 24 May 2024.
Builds on
20 of 46 references.
References · 20
- 2014OpenML: networked science in machine learningcited by 526
- 2015Doubly Robust Policy Evaluation and Optimizationcited by 158
- 2018More Robust Doubly Robust Off-policy Evaluationcited by 141
- 2019Empirical Study of Off-Policy Policy Evaluation for Reinforcement Learningcited by 121
- 2016Optimal and Adaptive Off-policy Evaluation in Contextual Banditscited by 111
- 2020The Importance of Pessimism in Fixed-Dataset Policy Optimizationcited by 99
- 2019Doubly robust off-policy evaluation with shrinkagecited by 87
- 2017The reparameterization trick for acquisition functionscited by 63
- 2018Policy Evaluation and Optimization with Continuous Treatmentscited by 58
- 2022Off-Policy Evaluation for Large Action Spaces via Embeddingscited by 54
- 2020Adaptive Estimator Selection for Off-Policy Evaluationcited by 38
- 2023Off-Policy Evaluation for Large Action Spaces via Conjunct Effect Modelingcited by 34
8 more
- 2021Evaluating the Robustness of Off-Policy Evaluationcited by 32
- 2022Policy-Adaptive Estimator Selection for Off-Policy Evaluationcited by 24
- 2023Exponential Smoothing for Off-Policy Learningcited by 15
- 2023Off-Policy Evaluation for Large Action Spaces via Policy Convolutioncited by 14
- 2022Data-Efficient Pipeline for Offline Reinforcement Learning with Limited Datacited by 12
- 2021Model Selection in Batch Policy Optimizationcited by 11
- 2023Learning Action Embeddings for Off-Policy Evaluationcited by 7
- 2024Hyperparameter Optimization Can Even be Harmful in Off-Policy Learning and How to Deal with Itcited by 5
Built on by
Citing papers per year: 2026: 3.
3 papers.