mlsanityData to 7 Oct 2026

Off-policy evaluation for slate recommendation

arXiv 1605.04812 ↗First posted 16 May 2016cs.LG cs.AI stat.MLcited by 102

Not on the map. Why?

Abstract (arXiv)

This paper studies the evaluation of policies that recommend an ordered set of items (e.g., a ranking) based on some context---a common scenario in web search, ads, and recommendation. We build on techniques from combinatorial bandits to introduce a new practical estimator that uses logged data to estimate a policy's performance. A thorough empirical evaluation on real-world data reveals that our estimator is accurate in a variety of settings, including as a subroutine in a learning-to-rank task, where it achieves competitive performance. We derive conditions under which our estimator is unbiased---these conditions are weaker than prior heuristics for slate evaluation---and experimentally demonstrate a smaller bias than parametric approaches, even when these conditions are violated. Finally, our theory and experiments also show exponential savings in the amount of required data compared with general unbiased estimators.

Key facts. Cited by 102 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 May 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 21; 2022: 23; 2023: 24; 2024: 11; 2025: 10; 2026: 12.

102 papers.

Sort:

By area:

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  2. 2021The Benchmark Lotterycited by 133
  3. 2021Benchmarks for Deep Off-Policy Evaluationcited by 89
  4. 2022Off-Policy Evaluation for Large Action Spaces via Embeddingscited by 53
  5. 2023Leveraging Factored Action Spaces for Efficient Offline Reinforcement Learning in Healthcarecited by 41
  6. 2023Off-Policy Evaluation for Large Action Spaces via Conjunct Effect Modelingcited by 34
  7. 2022Policy learning "without" overlap: Pessimism and generalized empirical Bernstein's inequalitycited by 32
  8. 2022Doubly Robust Off-Policy Evaluation for Ranking Policies under the Cascade Behavior Modelcited by 30
  9. 2022Conformal Off-Policy Prediction in Contextual Banditscited by 25
  10. 2021Policy Learning with Adaptively Collected Datacited by 21
  11. 2021A Deep Reinforcement Learning Approach to Marginalized Importance Sampling with the Successor Representationcited by 19
  12. 2022PAC-Bayesian Offline Contextual Bandits With Guaranteescited by 19
38 more
  1. 2024Off-Policy Evaluation of Slate Bandit Policies via Optimizing Abstractioncited by 17
  2. 2023Off-Policy Evaluation of Ranking Policies under Diverse User Behaviorcited by 16
  3. 2024Logarithmic Smoothing for Pessimistic Off-Policy Evaluation, Selection and Learningcited by 16
  4. 2022Safe Exploration for Efficient Policy Evaluation and Comparisoncited by 15
  5. 2021On component interactions in two-stage recommender systemscited by 14
  6. 2023Off-Policy Evaluation for Large Action Spaces via Policy Convolutioncited by 14
  7. 2021Control Variates for Slate Off-Policy Evaluationcited by 13
  8. 2023On (Normalised) Discounted Cumulative Gain as an Off-Policy Evaluation Metric for Top-n Recommendationcited by 13
  9. 2023GVFs in the Real World: Making Predictions Online for Water Treatmentcited by 13
  10. 2022A Behavior Regularized Implicit Policy for Offline Reinforcement Learningcited by 11
  11. 2021Variation Control and Evaluation for Generative SlateRecommendationscited by 10
  12. 2021Finding Valid Adjustments under Non-ignorability with Minimal DAG Knowledgecited by 10
  13. 2022Non-Linear Reinforcement Learning in Large Action Spaces: Structural Conditions and Sample-efficiency of Posterior Samplingcited by 10
  14. 2022A Survey of Deep Causal Models and Their Industrial Applicationscited by 10
  15. 2022Instrumental Variables in Causal Inference and Machine Learning: A Surveycited by 10
  16. 2023On the Opportunities and Challenges of Offline Reinforcement Learning for Recommender Systemscited by 10
  17. 2021Towards Content Provider Aware Recommender Systems: A Simulation Study on the Interplay between User and Provider Utilitiescited by 9
  18. 2021A Map of Bandits for E-commercecited by 9
  19. 2022Multi-Task Off-Policy Learning from Bandit Feedbackcited by 9
  20. 2023Practical Bandits: An Industry Perspectivecited by 9
  21. 2023Marginal Density Ratio for Off-Policy Evaluation in Contextual Banditscited by 9
  22. 2021Top-k eXtreme Contextual Bandits with Arm Hierarchycited by 8
  23. 2022Offline Reinforcement Learning for Mobile Notificationscited by 8
  24. 2024Effective Off-Policy Evaluation and Learning in Contextual Combinatorial Banditscited by 8
  25. 2021Combining Online Learning and Offline Learning for Contextual Bandits with Deficient Supportcited by 7
  26. 2021Contextual Bandit Applications in Customer Support Botcited by 7
  27. 2022Probabilistic Rank and Reward: A Scalable Model for Slate Recommendationcited by 7
  28. 2023Learning Action Embeddings for Off-Policy Evaluationcited by 7
  29. 2023Individualized Policy Evaluation and Learning under Clustered Network Interferencecited by 6
  30. 2024Bayesian Off-Policy Evaluation and Learning for Large Action Spacescited by 5
  31. 2021Combining Reward and Rank Signals for Slate Recommendationcited by 4
  32. 2021Value Penalized Q-Learning for Recommender Systemscited by 4
  33. 2021Doubly Robust Interval Estimation for Optimal Policy Evaluation in Online Learningcited by 4
  34. 2021Safe Data Collection for Offline and Online Policy Learningcited by 4
  35. 2022Scalable and Robust Self-Learning for Skill Routing in Large-Scale Conversational AI Systemscited by 4
  36. 2022Constrained Policy Optimization for Controlled Self-Learning in Conversational AI Systemscited by 4
  37. 2023Improved Policy Evaluation for Randomized Trials of Algorithmic Resource Allocationcited by 4
  38. 2023On the Sample Complexity of Vanilla Model-Based Offline Reinforcement Learning with Dependent Samplescited by 4

50 most cited of 102 All 102 (JSON)