mlsanityData to 7 Oct 2026

Bayesian Off-Policy Evaluation and Learning for Large Action Spaces

arXiv 2402.14664 ↗First posted 22 Feb 2024cs.LG cs.AI stat.MLcited by 5

Not on the map. Why?

Abstract (arXiv)

In interactive systems, actions are often correlated, presenting an opportunity for more sample-efficient off-policy evaluation (OPE) and learning (OPL) in large action spaces. We introduce a unified Bayesian framework to capture these correlations through structured and informative priors. In this framework, we propose sDM, a generic Bayesian approach for OPE and OPL, grounded in both algorithmic and theoretical foundations. Notably, sDM leverages action correlations without compromising computational efficiency. Moreover, inspired by online Bayesian bandits, we introduce Bayesian metrics that assess the average performance of algorithms across multiple problem instances, deviating from the conventional worst-case assessments. We analyze sDM in OPE and OPL, highlighting the benefits of leveraging action correlations. Empirical evidence showcases the strong performance of sDM.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 31 of its 75 references that are in the map. First posted on arXiv 22 Feb 2024.

Builds on

31 of 75 references.

References · 31

  1. 2017A Tutorial on Thompson Samplingcited by 420
  2. 2020Is Pessimism Provably Efficient for Offline RL?cited by 362
  3. 2013Learning to Optimize Via Posterior Samplingcited by 253
  4. 2015Doubly Robust Policy Evaluation and Optimizationcited by 158
  5. 2018More Robust Doubly Robust Off-policy Evaluationcited by 141
  6. 2016Optimal and Adaptive Off-policy Evaluation in Contextual Banditscited by 111
  7. 2016Off-policy evaluation for slate recommendationcited by 102
  8. 2018Offline A/B testing for Recommender Systemscited by 94
  9. 2022KuaiRec: A Fully-observed Dataset and Insights for Evaluating Recommender Systemscited by 94
  10. 2019Doubly robust off-policy evaluation with shrinkagecited by 87
  11. 2020Off-policy Bandits with Deficient Supportcited by 60
  12. 2022Off-Policy Evaluation for Large Action Spaces via Embeddingscited by 53
19 more
  1. 2016Cascading Bandits for Large-Scale Recommendation Problemscited by 45
  2. 2020Confident Off-Policy Evaluation and Selection through Self-Normalized Importance Weightingcited by 38
  3. 2018Importance Sampling Policy Evaluation with an Estimated Behavior Policycited by 34
  4. 2023Off-Policy Evaluation for Large Action Spaces via Conjunct Effect Modelingcited by 34
  5. 2022Contextual Bandits with Large Action Spaces: Made Practicalcited by 32
  6. 2018Bayesian Counterfactual Risk Minimizationcited by 28
  7. 2019Importance Resampling for Off-policy Predictioncited by 26
  8. 2020BLOB : A Probabilistic Model for Recommendation that Combines Organic and Bandit Signalscited by 19
  9. 2022PAC-Bayesian Offline Contextual Bandits With Guaranteescited by 19
  10. 2023Exponential Smoothing for Off-Policy Learningcited by 15
  11. 2022Mixed-Effect Thompson Samplingcited by 14
  12. 2023Oracle-Efficient Pessimism: Offline Policy Optimization in Contextual Banditscited by 14
  13. 2023Off-Policy Evaluation for Large Action Spaces via Policy Convolutioncited by 14
  14. 2022Multi-Task Off-Policy Learning from Bandit Feedbackcited by 9
  15. 2023Marginal Density Ratio for Off-Policy Evaluation in Contextual Banditscited by 9
  16. 2022Probabilistic Rank and Reward: A Scalable Model for Slate Recommendationcited by 7
  17. 2023Learning Action Embeddings for Off-Policy Evaluationcited by 7
  18. 2024Diffusion Models Meet Contextual Banditscited by 5
  19. 2023Value-aware Importance Weighting for Off-policy Reinforcement Learningcited by 4

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 3.

5 papers.

  1. 2024Logarithmic Smoothing for Pessimistic Off-Policy Evaluation, Selection and Learningcited by 16
  2. 2024Unified PAC-Bayesian Study of Pessimism for Offline Policy Learning with Regularized Importance Samplingcited by 3
  3. 2025Doubly Robust Fusion of Many Treatments for Policy Learningcited by 1
  4. 2025Off-Policy Learning in Large Action Spaces: Optimization Matters More Than Estimationcited by 0
  5. 2025Offline Contextual Bandit with Counterfactual Sample Identificationcited by 0