mlsanityData to 7 Oct 2026

Doubly robust off-policy evaluation with shrinkage

arXiv 1907.09623 ↗First posted 22 Jul 2019cs.LG stat.MLcited by 87

Not on the map. Why?

Abstract (arXiv)

We propose a new framework for designing estimators for off-policy evaluation in contextual bandits. Our approach is based on the asymptotically optimal doubly robust estimator, but we shrink the importance weights to minimize a bound on the mean squared error, which results in a better bias-variance tradeoff in finite samples. We use this optimization-based framework to obtain three estimators: (a) a weight-clipping estimator, (b) a new weight-shrinkage estimator, and (c) the first shrinkage-based estimator for combinatorial action sets. Extensive experiments in both standard and combinatorial bandit benchmark problems show that our estimators are highly adaptive and typically outperform state-of-the-art methods.

Key facts. Cited by 87 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Jul 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 11; 2022: 13; 2023: 15; 2024: 18; 2025: 14; 2026: 15.

87 papers.

Sort:

By area:

  1. 2022A Review of Off-Policy Evaluation in Reinforcement Learningcited by 92
  2. 2022Off-Policy Evaluation for Large Action Spaces via Embeddingscited by 53
  3. 2021Off-Policy Evaluation via Adaptive Weighting with Data from Contextual Banditscited by 40
  4. 2023Off-Policy Evaluation for Large Action Spaces via Conjunct Effect Modelingcited by 34
  5. 2021Evaluating the Robustness of Off-Policy Evaluationcited by 32
  6. 2022Minimax Regret Optimization for Robust Machine Learning under Distribution Shiftcited by 31
  7. 2022Doubly Robust Off-Policy Evaluation for Ranking Policies under the Cascade Behavior Modelcited by 30
  8. 2022Conformal Off-Policy Prediction in Contextual Banditscited by 25
  9. 2022Policy-Adaptive Estimator Selection for Off-Policy Evaluationcited by 24
  10. 2020Q* Approximation Schemes for Batch Reinforcement Learning: A Theoretical Comparisoncited by 22
  11. 2021Policy Learning with Adaptively Collected Datacited by 21
  12. 2024Off-Policy Evaluation of Slate Bandit Policies via Optimizing Abstractioncited by 17
38 more
  1. 2021Accelerating Offline Reinforcement Learning Application in Real-Time Bidding and Recommendation: Potential Use of Simulationcited by 16
  2. 2023Off-Policy Evaluation of Ranking Policies under Diverse User Behaviorcited by 16
  3. 2024Logarithmic Smoothing for Pessimistic Off-Policy Evaluation, Selection and Learningcited by 16
  4. 2022Safe Exploration for Efficient Policy Evaluation and Comparisoncited by 15
  5. 2023Exponential Smoothing for Off-Policy Learningcited by 15
  6. 2024Off-Dynamics Reinforcement Learning via Domain Adaptation and Reward Augmented Imitationcited by 15
  7. 2023Oracle-Efficient Pessimism: Offline Policy Optimization in Contextual Banditscited by 14
  8. 2023Off-Policy Evaluation for Large Action Spaces via Policy Convolutioncited by 14
  9. 2021Control Variates for Slate Off-Policy Evaluationcited by 13
  10. 2023On (Normalised) Discounted Cumulative Gain as an Off-Policy Evaluation Metric for Top-n Recommendationcited by 13
  11. 2024POTEC: Off-Policy Learning for Large Action Spaces via Two-Stage Policy Decompositioncited by 13
  12. 2024Optimal Baseline Corrections for Off-Policy Contextual Banditscited by 13
  13. 2023Ad-load Balancing via Off-policy Learning in a Content Marketplacecited by 11
  14. 2025Doubly Robust Alignment for Large Language Modelscited by 11
  15. 2023Marginal Density Ratio for Off-Policy Evaluation in Contextual Banditscited by 9
  16. 2024Long-term Off-Policy Evaluation and Learningcited by 9
  17. 2024Delta-rm OPE: Off-Policy Estimation with Pairs of Policiescited by 9
  18. 2024Effective Off-Policy Evaluation and Learning in Contextual Combinatorial Banditscited by 8
  19. 2021Optimal Model Averaging: Towards Personalized Collaborative Learningcited by 7
  20. 2022Off-Policy Risk Assessment in Markov Decision Processescited by 7
  21. 2023Learning Action Embeddings for Off-Policy Evaluationcited by 7
  22. 2024Bayesian Off-Policy Evaluation and Learning for Large Action Spacescited by 5
  23. 2024Hyperparameter Optimization Can Even be Harmful in Off-Policy Learning and How to Deal with Itcited by 5
  24. 2021Doubly Robust Interval Estimation for Optimal Policy Evaluation in Online Learningcited by 4
  25. 2022Fast Offline Policy Optimization for Large Scale Recommendationcited by 4
  26. 2024Distributionally Robust Policy Evaluation under General Covariate Shift in Contextual Banditscited by 4
  27. 2021State Relevance for Off-Policy Evaluationcited by 3
  28. 2022Counterfactual Learning with General Data-generating Policiescited by 3
  29. 2023SPEED: Experimental Design for Policy Evaluation in Linear Heteroscedastic Banditscited by 3
  30. 2023Uncertainty-Aware Instance Reweighting for Off-Policy Learningcited by 3
  31. 2023Does Sparsity Help in Learning Misspecified Linear Bandits?cited by 3
  32. 2023When is Off-Policy Evaluation (Reward Modeling) Useful in Contextual Bandits? A Data-Centric Perspectivecited by 3
  33. 2024Online Policy Learning and Inference by Matrix Completioncited by 3
  34. 2024Cross-Validated Off-Policy Evaluationcited by 3
  35. 2024Unified PAC-Bayesian Study of Pessimism for Offline Policy Learning with Regularized Importance Samplingcited by 3
  36. 2024Off-policy estimation with adaptively collected data: the power of online learningcited by 3
  37. 2021Off-Policy Evaluation of Slate Policies under Bayes Riskcited by 2
  38. 2022On the Reuse Bias in Off-Policy Reinforcement Learningcited by 2

50 most cited of 87 All 87 (JSON)