# Counterfactual Evaluation of Slate Recommendations with Sequential Reward Interactions

arXiv: https://arxiv.org/abs/2007.12986 · HTML: https://mlsanity.com/paper/2007.12986/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jul 2020.

## Built on by (most cited first, 30 of 35)

- [Advances and Challenges in Conversational Recommender Systems: A Survey](https://mlsanity.com/paper/2101.09459/) (2021)
- [Off-Policy Evaluation for Large Action Spaces via Embeddings](https://mlsanity.com/paper/2202.06317/) (2022)
- [Off-Policy Evaluation for Large Action Spaces via Conjunct Effect Modeling](https://mlsanity.com/paper/2305.08062/) (2023)
- [Doubly Robust Off-Policy Evaluation for Ranking Policies under the Cascade Behavior Model](https://mlsanity.com/paper/2202.01562/) (2022)
- [Off-Policy Evaluation of Slate Bandit Policies via Optimizing Abstraction](https://mlsanity.com/paper/2402.02171/) (2024)
- [Accelerating Offline Reinforcement Learning Application in Real-Time Bidding and Recommendation: Potential Use of Simulation](https://mlsanity.com/paper/2109.08331/) (2021)
- [Off-Policy Evaluation of Ranking Policies under Diverse User Behavior](https://mlsanity.com/paper/2306.15098/) (2023)
- [Causal Incremental Graph Convolution for Recommender System Retraining](https://mlsanity.com/paper/2108.06889/) (2021)
- [Control Variates for Slate Off-Policy Evaluation](https://mlsanity.com/paper/2106.07914/) (2021)
- [Generative Slate Recommendation with Reinforcement Learning](https://mlsanity.com/paper/2301.08632/) (2023)
- [On (Normalised) Discounted Cumulative Gain as an Off-Policy Evaluation Metric for Top-n Recommendation](https://mlsanity.com/paper/2307.15053/) (2023)
- [Variation Control and Evaluation for Generative SlateRecommendations](https://mlsanity.com/paper/2102.13302/) (2021)
- [Probabilistic Rank and Reward: A Scalable Model for Slate Recommendation](https://mlsanity.com/paper/2208.06263/) (2022)
- [Effective Off-Policy Evaluation and Learning in Contextual Combinatorial Bandits](https://mlsanity.com/paper/2408.11202/) (2024)
- [Hyperparameter Optimization Can Even be Harmful in Off-Policy Learning and How to Deal with It](https://mlsanity.com/paper/2404.15084/) (2024)
- [FinTRec: Transformer Based Unified Contextual Ads Targeting and Personalization for Financial Applications](https://mlsanity.com/paper/2511.14865/) (2025)
- [Off-Policy Evaluation of Slate Policies under Bayes Risk](https://mlsanity.com/paper/2101.02553/) (2021)
- [Off-Policy Evaluation in Embedded Spaces](https://mlsanity.com/paper/2203.02807/) (2022)
- [Pessimistic Off-Policy Optimization for Learning to Rank](https://mlsanity.com/paper/2206.02593/) (2022)
- [Offline Evaluation of Reward-Optimizing Recommender Systems: The Case of Simulation](https://mlsanity.com/paper/2209.08642/) (2022)
- [Distributional Off-Policy Evaluation for Slate Recommendations](https://mlsanity.com/paper/2308.14165/) (2023)
- [Algorithmic Fairness: A Tolerance Perspective](https://mlsanity.com/paper/2405.09543/) (2024)
- [Offline A/B Testing of Slate Recommendation Systems with LLMs: Reducing the Dependency on Pre-Collected User Interaction Data](https://mlsanity.com/paper/2511.04541/) (2025)
- [CaliCausalRank: Calibrated Multi-Objective Ad Ranking with Robust Counterfactual Utility Optimization](https://mlsanity.com/paper/2602.18786/) (2026)
- [Evaluating Uplift Modeling under Structural Biases: Insights into Metric Stability and Model Robustness](https://mlsanity.com/paper/2603.20775/) (2026)
- [Off-Policy Evaluation for Ranking Policies under Deterministic Logging Policies](https://mlsanity.com/paper/2603.21485/) (2026)
- [Inverse Propensity Score based offline estimator for deterministic ranking lists using position bias](https://mlsanity.com/paper/2208.14980/) (2022)
- [Finding Interest Needle in Popularity Haystack: Improving Retrieval by Modeling Item Exposure](https://mlsanity.com/paper/2503.23630/) (2025)
- [Off-Policy Evaluation of Ranking Policies via Embedding-Space User Behavior Modeling](https://mlsanity.com/paper/2506.00446/) (2025)
- [Identifying Offline Metrics that Predict Online Impact: A Pragmatic Strategy for Real-World Recommender Systems](https://mlsanity.com/paper/2507.09566/) (2025)

## Abstract (arXiv)

Users of music streaming, video streaming, news recommendation, and e-commerce services often engage with content in a sequential manner. Providing and evaluating good sequences of recommendations is therefore a central problem for these services. Prior reweighting-based counterfactual evaluation methods either suffer from high variance or make strong independence assumptions about rewards. We propose a new counterfactual estimator that allows for sequential interactions in the rewards with lower variance in an asymptotically unbiased manner. Our method uses graphical assumptions about the causal relationships of the slate to reweight the rewards in the logging policy in a way that approximates the expected sum of rewards under the target policy. Extensive experiments in simulation and on a live recommender system show that our approach outperforms existing methods in terms of bias and data efficiency for the sequential track recommendations problem.