mlsanityData to 14 Oct 2026

Safe Exploration for Efficient Policy Evaluation and Comparison

arXiv 2202.13234 ↗First posted 26 Feb 2022cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

High-quality data plays a central role in ensuring the accuracy of policy evaluation. This paper initiates the study of efficient and safe data collection for bandit policy evaluation. We formulate the problem and investigate its several representative variants. For each variant, we analyze its statistical properties, derive the corresponding exploration policy, and design an efficient algorithm for computing it. Both theoretical analysis and experiments support the usefulness of the proposed methods.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 25 of its 55 references that are in the map. First posted on arXiv 26 Feb 2022.

Builds on

25 of 55 references.

References · 25

  1. 2021Convex optimizationcited by 2,515
  2. 2019Introduction to Multi-Armed Banditscited by 481
  3. 2015Doubly Robust Off-policy Value Evaluation for Reinforcement Learningcited by 339
  4. 2015Doubly Robust Policy Evaluation and Optimizationcited by 158
  5. 2016Optimal and Adaptive Off-policy Evaluation in Contextual Banditscited by 111
  6. 2016Off-policy evaluation for slate recommendationcited by 103
  7. 2019Linear Stochastic Bandits Under Safety Constraintscited by 90
  8. 2019Doubly robust off-policy evaluation with shrinkagecited by 87
  9. 2019Non-Asymptotic Pure Exploration by Solving Gamescited by 63
  10. 2021Meta-Thompson Samplingcited by 63
  11. 2020Off-policy Bandits with Deficient Supportcited by 61
  12. 2016Conservative Banditscited by 59
13 more
  1. 2016Conservative Contextual Linear Banditscited by 57
  2. 2017Fully adaptive algorithm for pure exploration in linear banditscited by 43
  3. 2015Residual Weighted Learning for Estimating Individualized Treatment Rulescited by 38
  4. 2021Deeply-Debiased Off-Policy Interval Estimationcited by 36
  5. 2020Optimal Off-Policy Evaluation from Multiple Logging Policiescited by 30
  6. 2015Algorithms with Logarithmic or Sublinear Regret for Constrained Contextual Banditscited by 29
  7. 2020Stage-wise Conservative Linear Banditscited by 28
  8. 2021Metadata-based Multi-Task Bandits with Bayesian Hierarchical Modelscited by 24
  9. 2021Design of Experiments for Stochastic Contextual Linear Banditscited by 18
  10. 2020Deep Jump Learning for Off-Policy Evaluation in Continuous Treatment Settingscited by 14
  11. 2020Taking the Counterfactual Online: Efficient and Unbiased Online Evaluation for Rankingcited by 8
  12. 2021Combining Online Learning and Offline Learning for Contextual Bandits with Deficient Supportcited by 7
  13. 2022Variance-Optimal Augmentation Logging for Counterfactual Evaluation in Contextual Banditscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 7; 2024: 3; 2025: 3; 2026: 2.

15 papers.

Sort:

By area:

  1. 2023Pure Exploration in Bandits with Linear Constraintscited by 10
  2. 2023Directional Optimism for Safe Linear Banditscited by 8
  3. 2023On-Policy Policy Gradient Reinforcement Learning Without On-Policy Samplingcited by 6
  4. 2024Unraveling the Interplay between Carryover Effects and Reward Autocorrelations in Switchback Experimentscited by 6
  5. 2023Bridging Imitation and Online Reinforcement Learning: An Optimistic Talecited by 5
  6. 2024Experiment Planning with Function Approximationcited by 5
  7. 2025Balancing Interference and Correlation in Spatial Experimental Designs: A Causal Graph Cut Approachcited by 5
  8. 2023When is Off-Policy Evaluation (Reward Modeling) Useful in Contextual Bandits? A Data-Centric Perspectivecited by 4
  9. 2023SPEED: Experimental Design for Policy Evaluation in Linear Heteroscedastic Banditscited by 3
  10. 2023Efficient Online Learning with Offline Datasets for Infinite Horizon MDPs: A Bayesian Approachcited by 3
  11. 2024SaVeR: Optimal Data Collection Strategy for Safe Policy Evaluation in Tabular MDPcited by 3
  12. 2025Demystifying the Paradox of Importance Sampling with an Estimated History-Dependent Behavior Policy in Off-Policy Evaluationcited by 1
3 more
  1. 2026Designing Time Series Experiments in A/B Testing with Transformer Reinforcement Learningcited by 1
  2. 2025Centralized Adaptive Sampling for Reliable Co-Training of Independent Multi-Agent Policiescited by 0
  3. 2026Fast A/B/n Testing: Exact Multi-Policy Comparison via Tree-Coupled Feedback Sharingcited by 0