mlsanityData to 7 Oct 2026

Offline A/B testing for Recommender Systems

arXiv 1801.07030 ↗First posted 22 Jan 2018stat.ML cs.LGcited by 94

Not on the map. Why?

Abstract (arXiv)

Before A/B testing online a new version of a recommender system, it is usual to perform some offline evaluations on historical data. We focus on evaluation methods that compute an estimator of the potential uplift in revenue that could generate this new technology. It helps to iterate faster and to avoid losing money by detecting poor policies. These estimators are known as counterfactual or off-policy estimators. We show that traditional counterfactual estimators such as capped importance sampling and normalised importance sampling are experimentally not having satisfying bias-variance compromises in the context of personalised product recommendation for online advertising. We propose two variants of counterfactual estimates with different modelling of the bias that prove to be accurate in real-world conditions. We provide a benchmark of these estimators by showing their correlation with business metrics observed by running online A/B tests on a commercial recommender system.

Key facts. Cited by 94 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Jan 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 12; 2022: 17; 2023: 17; 2024: 12; 2025: 17; 2026: 18.

94 papers.

Sort:

By area:

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  2. 2021Advances and Challenges in Conversational Recommender Systems: A Surveycited by 118
  3. 2021Benchmarks for Deep Off-Policy Evaluationcited by 89
  4. 2021Understanding the role of importance weighting for deep learningcited by 38
  5. 2021Evaluating the Robustness of Off-Policy Evaluationcited by 32
  6. 2022Doubly Robust Off-Policy Evaluation for Ranking Policies under the Cascade Behavior Modelcited by 30
  7. 2022Policy-Adaptive Estimator Selection for Off-Policy Evaluationcited by 24
  8. 2022PrefRec: Recommender Systems with Human Preferences for Reinforcing Long-term User Engagementcited by 21
  9. 2022PAC-Bayesian Offline Contextual Bandits With Guaranteescited by 19
  10. 2025AgentA/B: Automated and Scalable Web A/BTesting with Interactive LLM Agentscited by 19
  11. 2023Balancing Unobserved Confounding with a Few Unbiased Ratings in Debiased Recommendationscited by 18
  12. 2021Accelerating Offline Reinforcement Learning Application in Real-Time Bidding and Recommendation: Potential Use of Simulationcited by 16
38 more
  1. 2023Off-Policy Evaluation of Ranking Policies under Diverse User Behaviorcited by 16
  2. 2024Logarithmic Smoothing for Pessimistic Off-Policy Evaluation, Selection and Learningcited by 16
  3. 2023Exponential Smoothing for Off-Policy Learningcited by 15
  4. 2022Regularizing a Model-based Policy Stationary Distribution to Stabilize Offline Reinforcement Learningcited by 14
  5. 2023On (Normalised) Discounted Cumulative Gain as an Off-Policy Evaluation Metric for Top-n Recommendationcited by 13
  6. 2023Correcting for Interference in Experiments: A Case Study at Douyincited by 12
  7. 2022A Behavior Regularized Implicit Policy for Offline Reinforcement Learningcited by 11
  8. 2022ResAct: Reinforcing Long-term Engagement in Sequential Recommendation with Residual Actorcited by 10
  9. 2024Multi-Objective Recommendation via Multivariate Policy Learningcited by 10
  10. 2026PolicySim: An LLM-Based Agent Social Simulation Sandbox for Proactive Policy Optimizationcited by 10
  11. 2024Delta-rm OPE: Off-Policy Estimation with Pairs of Policiescited by 9
  12. 2021Rethinking Neural vs. Matrix-Factorization Collaborative Filtering: the Theoretical Perspectivescited by 8
  13. 2022Local Policy Improvement for Recommender Systemscited by 8
  14. 2023Online Matching: A Real-time Bandit System for Large-scale Recommendationscited by 8
  15. 2022Probabilistic Rank and Reward: A Scalable Model for Slate Recommendationcited by 7
  16. 2021Simulations in Recommender Systems: An industry perspectivecited by 6
  17. 2022Meta Policy Learning for Cold-Start Conversational Recommendationcited by 6
  18. 2022Simpson's Paradox in Recommender Fairness: Reconciling differences between per-user and aggregated evaluationscited by 5
  19. 2023A Common Misassumption in Online Experiments with Machine Learning Modelscited by 5
  20. 2023Generative User-Experience Research for Developing Domain-specific Natural Language Processing Applicationscited by 5
  21. 2024Bayesian Off-Policy Evaluation and Learning for Large Action Spacescited by 5
  22. 2023On the Sample Complexity of Vanilla Model-Based Offline Reinforcement Learning with Dependent Samplescited by 4
  23. 2026Let the Agent Steer: Closed-Loop Ranking Optimization via Influence Exchangecited by 4
  24. 2022Bilateral Self-unbiased Learning from Biased Implicit Feedbackcited by 3
  25. 2024Unified PAC-Bayesian Study of Pessimism for Offline Policy Learning with Regularized Importance Samplingcited by 3
  26. 2025A Comprehensive Review on Harnessing Large Language Models to Overcome Recommender System Challengescited by 3
  27. 2026Exploring Recommender System Evaluation: A Multi-Modal User Agent Framework for A/B Testingcited by 3
  28. 2021Off-Policy Evaluation of Slate Policies under Bayes Riskcited by 2
  29. 2022Pessimistic Off-Policy Optimization for Learning to Rankcited by 2
  30. 2023Scalable End-to-End ML Platforms: from AutoML to Self-servecited by 2
  31. 2023Learning to Control Autonomous Fleets from Observation via Offline Reinforcement Learningcited by 2
  32. 2023Uncertainty Calibration for Counterfactual Propensity Estimation in Recommendationcited by 2
  33. 2023Improving the Knowledge Gradient Algorithmcited by 2
  34. 2024Automated Off-Policy Estimator Selection via Supervised Learningcited by 2
  35. 2025Off-Policy Evaluation and Learning for Matching Marketscited by 2
  36. 2025Harnessing the Power of Interleaving and Counterfactual Evaluation for Airbnb Search Rankingcited by 2
  37. 2025Mirroring Users: Towards Building Preference-aligned User Simulator with User Feedback in Recommendationcited by 2
  38. 2025Breaking Determinism: Stochastic Modeling for Reliable Off-Policy Evaluation in Ad Auctionscited by 2

50 most cited of 94 All 94 (JSON)