mlsanityData to 14 Oct 2026

Confident Off-Policy Evaluation and Selection through Self-Normalized Importance Weighting

arXiv 2006.10460 ↗First posted 18 Jun 2020cs.LG stat.MLcited by 38

Not on the map. Why?

Abstract (arXiv)

We consider off-policy evaluation in the contextual bandit setting for the purpose of obtaining a robust off-policy selection strategy, where the selection strategy is evaluated based on the value of the chosen policy in a set of proposal (target) policies. We propose a new method to compute a lower bound on the value of an arbitrary target policy given some logged data in contextual bandits for a desired coverage. The lower bound is built around the so-called Self-normalized Importance Weighting (SN) estimator. It combines the use of a semi-empirical Efron-Stein tail inequality to control the concentration and a new multiplicative (rather than additive) control of the bias. The new approach is evaluated on a number of synthetic and real datasets and is found to be superior to its main competitors, both in terms of tightness of the confidence intervals and the quality of the policies chosen.

Key facts. Cited by 38 papers in the map (data through 14 Oct 2026). First posted on arXiv 18 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 8; 2022: 13; 2023: 4; 2024: 6; 2025: 4; 2026: 3.

38 papers.

Sort:

By area:

  1. 2022A Review of Off-Policy Evaluation in Reinforcement Learningcited by 95
  2. 2022Why So Pessimistic? Estimating Uncertainties for Offline RL through Ensembles, and Why Their Independence Matterscited by 76
  3. 2021Model Selection for Offline Reinforcement Learning: Practical Considerations for Healthcare Settingscited by 68
  4. 2021Universal Off-Policy Evaluationcited by 43
  5. 2021Bootstrapping Fitted Q-Evaluation for Off-Policy Inferencecited by 40
  6. 2021A Minimax Learning Approach to Off-Policy Evaluation in Confounded Partially Observable Markov Decision Processescited by 36
  7. 2022Policy learning "without" overlap: Pessimism and generalized empirical Bernstein's inequalitycited by 32
  8. 2021Online Bootstrap Inference For Policy Evaluation in Reinforcement Learningcited by 26
  9. 2022Conformal Off-Policy Prediction in Contextual Banditscited by 25
  10. 2022Policy-Adaptive Estimator Selection for Off-Policy Evaluationcited by 24
  11. 2022PAC-Bayesian Offline Contextual Bandits With Guaranteescited by 20
  12. 2024Logarithmic Smoothing for Pessimistic Off-Policy Evaluation, Selection and Learningcited by 16
26 more
  1. 2023Exponential Smoothing for Off-Policy Learningcited by 15
  2. 2021Optimal Uniform OPE and Model-based Offline Reinforcement Learning in Time-Homogeneous, Reward-Free and Task-Agnostic Settingscited by 14
  3. 2022BR-SNIS: Bias Reduced Self-Normalized Importance Samplingcited by 12
  4. 2022Uncertainty Quantification for Fairness in Two-Stage Recommender Systemscited by 10
  5. 2024Target Networks and Over-parameterization Stabilize Off-policy Bootstrapping with Function Approximationcited by 10
  6. 2021High-Confidence Off-Policy (or Counterfactual) Variance Estimationcited by 8
  7. 2022PAC-Bayes Bounds for Bandit Problems: A Survey and Experimental Comparisoncited by 7
  8. 2023Conformal Off-Policy Evaluation in Markov Decision Processescited by 6
  9. 2024Bayesian Off-Policy Evaluation and Learning for Large Action Spacescited by 5
  10. 2024Hyperparameter Optimization Can Even be Harmful in Off-Policy Learning and How to Deal with Itcited by 5
  11. 2025Balancing Interference and Correlation in Spatial Experimental Designs: A Causal Graph Cut Approachcited by 5
  12. 2025Improved Offline Contextual Bandits with Second-Order Bounds: Betting and Freezingcited by 4
  13. 2021Pessimistic Model Selection for Offline Deep Reinforcement Learningcited by 3
  14. 2022Counterfactual Learning with General Data-generating Policiescited by 3
  15. 2023Probabilistic Offline Policy Ranking with Approximate Bayesian Computationcited by 3
  16. 2024Unified PAC-Bayesian Study of Pessimism for Offline Policy Learning with Regularized Importance Samplingcited by 3
  17. 2022On the Reuse Bias in Off-Policy Reinforcement Learningcited by 2
  18. 2022Semi-supervised Batch Learning From Logged Datacited by 2
  19. 2024Predictive Performance Comparison of Decision Policies Under Confoundingcited by 2
  20. 2025Clustering Context in Off-Policy Evaluationcited by 2
  21. 2023Asymptotically Unbiased Off-Policy Policy Evaluation when Reusing Old Data in Nonstationary Environmentscited by 1
  22. 2026Anytime-valid Optimal Policy Identificationcited by 1
  23. 2022Offline Policy Comparison with Confidence: Benchmarks and Baselinescited by 0
  24. 2025Clustered KL-barycenter design for policy evaluationcited by 0
  25. 2026Kernel weighted importance sampling for off-policy evaluation in contextual banditscited by 0
  26. 2026Fast A/B/n Testing: Exact Multi-Policy Comparison via Tree-Coupled Feedback Sharingcited by 0