mlsanityData to 14 Oct 2026

Adaptive Estimator Selection for Off-Policy Evaluation

arXiv 2002.07729 ↗First posted 18 Feb 2020cs.LG math.ST stat.MLcited by 38

Not on the map. Why?

Abstract (arXiv)

We develop a generic data-driven method for estimator selection in off-policy policy evaluation settings. We establish a strong performance guarantee for the method, showing that it is competitive with the oracle estimator, up to a constant factor. Via in-depth case studies in contextual bandits and reinforcement learning, we demonstrate the generality and applicability of the method. We also perform comprehensive experiments, demonstrating the empirical efficacy of our approach and comparing with related approaches. In both case studies, our method compares favorably with existing methods.

Key facts. Cited by 38 papers in the map (data through 14 Oct 2026). First posted on arXiv 18 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 6; 2022: 6; 2023: 9; 2024: 9; 2025: 5; 2026: 3.

38 papers.

Sort:

By area:

  1. 2022Off-Policy Evaluation for Large Action Spaces via Embeddingscited by 54
  2. 2021Deeply-Debiased Off-Policy Interval Estimationcited by 36
  3. 2023Off-Policy Evaluation for Large Action Spaces via Conjunct Effect Modelingcited by 34
  4. 2021Evaluating the Robustness of Off-Policy Evaluationcited by 32
  5. 2021Off-Policy Evaluation in Partially Observed Markov Decision Processes under Sequential Ignorabilitycited by 24
  6. 2022Policy-Adaptive Estimator Selection for Off-Policy Evaluationcited by 24
  7. 2024Off-Policy Evaluation of Slate Bandit Policies via Optimizing Abstractioncited by 17
  8. 2023Off-Policy Evaluation for Large Action Spaces via Policy Convolutioncited by 14
  9. 2022Oracle Inequalities for Model Selection in Offline Reinforcement Learningcited by 13
  10. 2023On (Normalised) Discounted Cumulative Gain as an Off-Policy Evaluation Metric for Top-n Recommendationcited by 13
  11. 2023Towards Assessing and Benchmarking Risk-Return Tradeoff of Off-Policy Evaluationcited by 13
  12. 2024POTEC: Off-Policy Learning for Large Action Spaces via Two-Stage Policy Decompositioncited by 13
26 more
  1. 2022Data-Efficient Pipeline for Offline Reinforcement Learning with Limited Datacited by 12
  2. 2021Model Selection in Batch Policy Optimizationcited by 11
  3. 2024Effective Off-Policy Evaluation and Learning in Contextual Combinatorial Banditscited by 8
  4. 2022Local Metric Learning for Off-Policy Evaluation in Contextual Bandits with Continuous Actionscited by 6
  5. 2023SCOPE-RL: A Python Library for Offline Reinforcement Learning and Off-Policy Evaluationcited by 6
  6. 2025Model Selection for Off-policy Evaluation: New Algorithms and Experimental Protocolcited by 6
  7. 2023When is Off-Policy Evaluation (Reward Modeling) Useful in Contextual Bandits? A Data-Centric Perspectivecited by 4
  8. 2021Pessimistic Model Selection for Offline Deep Reinforcement Learningcited by 3
  9. 2023When is Offline Policy Selection Sample Efficient for Reinforcement Learning?cited by 3
  10. 2024Cross-Validated Off-Policy Evaluationcited by 3
  11. 2024OPERA: Automatic Offline Policy Evaluation with Re-weighted Aggregates of Multiple Estimatorscited by 3
  12. 2024Kernel Metric Learning for In-Sample Off-Policy Evaluation of Deterministic RL Policiescited by 3
  13. 2024Abstract Reward Processes: Leveraging State Abstraction for Consistent Off-Policy Evaluationcited by 3
  14. 2021Improving Long-Term Metrics in Recommendation Systems using Short-Horizon Reinforcement Learningcited by 2
  15. 2023Double Clipping: Less-Biased Variance Reduction in Off-Policy Evaluationcited by 2
  16. 2024Automated Off-Policy Estimator Selection via Supervised Learningcited by 2
  17. 2025Clustering Context in Off-Policy Evaluationcited by 2
  18. 2025Off-Policy Evaluation and Learning for Matching Marketscited by 2
  19. 2022Off-policy evaluation for learning-to-rank via interpolating the item-position model and the position-based modelcited by 1
  20. 2023Doubly Robust Estimator for Off-Policy Evaluation with Large Action Spacescited by 1
  21. 2025A General Framework for Off-Policy Learning with Partially-Observed Rewardcited by 1
  22. 2026Off-Policy Evaluation for Semantic ID Recommenders: Does the Model's Own Code Hierarchy Help?cited by 1
  23. 2024Off-Policy Selection for Initiating Human-Centric Experimental Designcited by 0
  24. 2025Off-Policy Evaluation of Ranking Policies via Embedding-Space User Behavior Modelingcited by 0
  25. 2026Detecting LLM-Generated Tokens in Human--LLM Coauthored Textcited by 0
  26. 2026Cross-Domain Off-Policy Evaluation and Learning for Contextual Banditscited by 0