# Debiased Off-Policy Evaluation for Recommendation Systems

arXiv: https://arxiv.org/abs/2002.08536 · HTML: https://mlsanity.com/paper/2002.08536/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Feb 2020.

## Built on by (most cited first, 9 of 9)

- [Policy-Adaptive Estimator Selection for Off-Policy Evaluation](https://mlsanity.com/paper/2211.13904/) (2022)
- [DoubleML -- An Object-Oriented Implementation of Double Machine Learning in Python](https://mlsanity.com/paper/2104.03220/) (2021)
- [On the Opportunities and Challenges of Offline Reinforcement Learning for Recommender Systems](https://mlsanity.com/paper/2308.11336/) (2023)
- [Longitudinal Targeted Minimum Loss-based Estimation with Temporal-Difference Heterogeneous Transformer](https://mlsanity.com/paper/2404.04399/) (2024)
- [Automated Off-Policy Estimator Selection via Supervised Learning](https://mlsanity.com/paper/2406.18022/) (2024)
- [Breaking Determinism: Stochastic Modeling for Reliable Off-Policy Evaluation in Ad Auctions](https://mlsanity.com/paper/2512.03354/) (2025)
- [Doubly Robust Estimator for Off-Policy Evaluation with Large Action Spaces](https://mlsanity.com/paper/2308.03443/) (2023)
- [Identifying Offline Metrics that Predict Online Impact: A Pragmatic Strategy for Real-World Recommender Systems](https://mlsanity.com/paper/2507.09566/) (2025)
- [Latent Variable Modeling for Robust Causal Effect Estimation](https://mlsanity.com/paper/2508.20259/) (2025)

## Abstract (arXiv)

Efficient methods to evaluate new algorithms are critical for improving interactive bandit and reinforcement learning systems such as recommendation systems. A/B tests are reliable, but are time- and money-consuming, and entail a risk of failure. In this paper, we develop an alternative method, which predicts the performance of algorithms given historical data that may have been generated by a different algorithm. Our estimator has the property that its prediction converges in probability to the true performance of a counterfactual algorithm at a rate of √N, as the sample size N increases. We also show a correct way to estimate the variance of our prediction, thus allowing the analyst to quantify the uncertainty in the prediction. These properties hold even when the analyst does not know which among a large number of potentially important state variables are actually important. We validate our method by a simulation experiment about reinforcement learning. We finally apply it to improve advertisement design by a major advertisement company. We find that our method produces smaller mean squared errors than state-of-the-art methods.