# Control Variates for Slate Off-Policy Evaluation

arXiv: https://arxiv.org/abs/2106.07914 · HTML: https://mlsanity.com/paper/2106.07914/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 45 references that are in the map.
- First posted on arXiv 15 Jun 2021.

## Builds on (most cited first, 15 of 15)

- [Data-Efficient Off-Policy Policy Evaluation for Reinforcement Learning](https://mlsanity.com/paper/1604.00923/) (2016)
- [Top-K Off-Policy Correction for a REINFORCE Recommender System](https://mlsanity.com/paper/1812.02353/) (2018)
- [More Robust Doubly Robust Off-policy Evaluation](https://mlsanity.com/paper/1802.03493/) (2018)
- [Introducing LETOR 4.0 Datasets](https://mlsanity.com/paper/1306.2597/) (2013)
- [Optimal and Adaptive Off-policy Evaluation in Contextual Bandits](https://mlsanity.com/paper/1612.01205/) (2016)
- [Off-policy evaluation for slate recommendation](https://mlsanity.com/paper/1605.04812/) (2016)
- [Doubly robust off-policy evaluation with shrinkage](https://mlsanity.com/paper/1907.09623/) (2019)
- [Counterfactual Evaluation of Slate Recommendations with Sequential Reward Interactions](https://mlsanity.com/paper/2007.12986/) (2020)
- [An Efficient Bandit Algorithm for Realtime Multivariate Optimization](https://mlsanity.com/paper/1810.09558/) (2018)
- [Offline Evaluation of Ranking Policies with Click Models](https://mlsanity.com/paper/1804.10488/) (2018)
- [Intrinsically Efficient, Stable, and Bounded Off-Policy Evaluation for Reinforcement Learning](https://mlsanity.com/paper/1906.03735/) (2019)
- [Policy Learning with Observational Data](https://mlsanity.com/paper/1702.02896/) (2017)
- [More Efficient Off-Policy Evaluation through Regularized Targeted Learning](https://mlsanity.com/paper/1912.06292/) (2019)
- [Learning from eXtreme Bandit Feedback](https://mlsanity.com/paper/2009.12947/) (2020)
- [Offline Comparison of Ranking Functions using Randomized Data](https://mlsanity.com/paper/1810.05252/) (2018)

## Built on by (most cited first, 13 of 13)

- [Off-Policy Evaluation for Large Action Spaces via Embeddings](https://mlsanity.com/paper/2202.06317/) (2022)
- [Off-Policy Evaluation for Large Action Spaces via Conjunct Effect Modeling](https://mlsanity.com/paper/2305.08062/) (2023)
- [Efficient Attention via Control Variates](https://mlsanity.com/paper/2302.04542/) (2023)
- [Effective Off-Policy Evaluation and Learning in Contextual Combinatorial Bandits](https://mlsanity.com/paper/2408.11202/) (2024)
- [Exploiting Correlated Auxiliary Feedback in Parameterized Bandits](https://mlsanity.com/paper/2311.02715/) (2023)
- [Safe Data Collection for Offline and Online Policy Learning](https://mlsanity.com/paper/2111.04835/) (2021)
- [Distributional Off-Policy Evaluation for Slate Recommendations](https://mlsanity.com/paper/2308.14165/) (2023)
- [Efficient Algorithms for Logistic Contextual Slate Bandits with Bandit Feedback](https://mlsanity.com/paper/2506.13163/) (2025)
- [Additive Control Variates Dominate Self-Normalisation in Off-Policy Evaluation](https://mlsanity.com/paper/2602.14914/) (2026)
- [Quotient DAGs for Off-Policy Evaluation:Forward-Flow Importance Sampling and Exact Slate Propensities](https://mlsanity.com/paper/2605.29500/) (2026)
- [CASP: Support-Aware Offline Policy Selection for Two-Stage Recommender Systems](https://mlsanity.com/paper/2604.23022/) (2026)
- [Offline Contextual Bandits in the Presence of New Actions](https://mlsanity.com/paper/2605.18509/) (2026)
- [Recommendation Ranking Off-Policy Evaluation under Ranking-Dependent Examination via Examination-Relevance Decomposition](https://mlsanity.com/paper/2609.35034/) (2026)

## Abstract (arXiv)

We study the problem of off-policy evaluation from batched contextual bandit data with multidimensional actions, often termed slates. The problem is common to recommender systems and user-interface optimization, and it is particularly challenging because of the combinatorially-sized action space. Swaminathan et al. (2017) have proposed the pseudoinverse (PI) estimator under the assumption that the conditional mean rewards are additive in actions. Using control variates, we consider a large class of unbiased estimators that includes as specific cases the PI estimator and (asymptotically) its self-normalized variant. By optimizing over this class, we obtain new estimators with risk improvement guarantees over both the PI and the self-normalized PI estimators. Experiments with real-world recommender data as well as synthetic data validate these improvements in practice.