# Doubly Robust Off-Policy Evaluation for Ranking Policies under the Cascade Behavior Model

arXiv: https://arxiv.org/abs/2202.01562 · HTML: https://mlsanity.com/paper/2202.01562/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 25 references that are in the map.
- First posted on arXiv 3 Feb 2022.

## Builds on (most cited first, 16 of 16)

- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Doubly Robust Off-policy Value Evaluation for Reinforcement Learning](https://mlsanity.com/paper/1511.03722/) (2015)
- [Data-Efficient Off-Policy Policy Evaluation for Reinforcement Learning](https://mlsanity.com/paper/1604.00923/) (2016)
- [Doubly Robust Policy Evaluation and Optimization](https://mlsanity.com/paper/1503.02834/) (2015)
- [More Robust Doubly Robust Off-policy Evaluation](https://mlsanity.com/paper/1802.03493/) (2018)
- [Off-policy evaluation for slate recommendation](https://mlsanity.com/paper/1605.04812/) (2016)
- [Offline A/B testing for Recommender Systems](https://mlsanity.com/paper/1801.07030/) (2018)
- [Doubly robust off-policy evaluation with shrinkage](https://mlsanity.com/paper/1907.09623/) (2019)
- [Unbiased Recommender Learning from Missing-Not-At-Random Implicit Feedback](https://mlsanity.com/paper/1909.03601/) (2019)
- [Open Bandit Dataset and Pipeline: Towards Realistic and Reproducible Off-Policy Evaluation](https://mlsanity.com/paper/2008.07146/) (2020)
- [Counterfactual Evaluation of Slate Recommendations with Sequential Reward Interactions](https://mlsanity.com/paper/2007.12986/) (2020)
- [Offline Evaluation of Ranking Policies with Click Models](https://mlsanity.com/paper/1804.10488/) (2018)
- [Evaluating the Robustness of Off-Policy Evaluation](https://mlsanity.com/paper/2108.13703/) (2021)
- [Optimal Off-Policy Evaluation from Multiple Logging Policies](https://mlsanity.com/paper/2010.11002/) (2020)
- [Accelerating Offline Reinforcement Learning Application in Real-Time Bidding and Recommendation: Potential Use of Simulation](https://mlsanity.com/paper/2109.08331/) (2021)
- [Off-Policy Evaluation of Slate Policies under Bayes Risk](https://mlsanity.com/paper/2101.02553/) (2021)

## Built on by (most cited first, 30 of 30)

- [Off-Policy Evaluation for Large Action Spaces via Embeddings](https://mlsanity.com/paper/2202.06317/) (2022)
- [Off-Policy Evaluation for Large Action Spaces via Conjunct Effect Modeling](https://mlsanity.com/paper/2305.08062/) (2023)
- [Policy-Adaptive Estimator Selection for Off-Policy Evaluation](https://mlsanity.com/paper/2211.13904/) (2022)
- [An Instrumental Variable Approach to Confounded Off-Policy Evaluation](https://mlsanity.com/paper/2212.14468/) (2022)
- [Doubly-Robust Estimation for Correcting Position-Bias in Click Feedback for Unbiased Learning to Rank](https://mlsanity.com/paper/2203.17118/) (2022)
- [Off-Policy Evaluation of Slate Bandit Policies via Optimizing Abstraction](https://mlsanity.com/paper/2402.02171/) (2024)
- [Off-Policy Evaluation of Ranking Policies under Diverse User Behavior](https://mlsanity.com/paper/2306.15098/) (2023)
- [On (Normalised) Discounted Cumulative Gain as an Off-Policy Evaluation Metric for Top-n Recommendation](https://mlsanity.com/paper/2307.15053/) (2023)
- [Towards Assessing and Benchmarking Risk-Return Tradeoff of Off-Policy Evaluation](https://mlsanity.com/paper/2311.18207/) (2023)
- [POTEC: Off-Policy Learning for Large Action Spaces via Two-Stage Policy Decomposition](https://mlsanity.com/paper/2402.06151/) (2024)
- [Multiple Robust Learning for Recommendation](https://mlsanity.com/paper/2207.10796/) (2022)
- [Practical Bandits: An Industry Perspective](https://mlsanity.com/paper/2302.01223/) (2023)
- [Long-term Off-Policy Evaluation and Learning](https://mlsanity.com/paper/2404.15691/) (2024)
- [Effective Off-Policy Evaluation and Learning in Contextual Combinatorial Bandits](https://mlsanity.com/paper/2408.11202/) (2024)
- [SCOPE-RL: A Python Library for Offline Reinforcement Learning and Off-Policy Evaluation](https://mlsanity.com/paper/2311.18206/) (2023)
- [Hyperparameter Optimization Can Even be Harmful in Off-Policy Learning and How to Deal with It](https://mlsanity.com/paper/2404.15084/) (2024)
- [Unified Off-Policy Learning to Rank: a Reinforcement Learning Perspective](https://mlsanity.com/paper/2306.07528/) (2023)
- [Pessimistic Off-Policy Optimization for Learning to Rank](https://mlsanity.com/paper/2206.02593/) (2022)
- [Distributional Off-Policy Evaluation for Slate Recommendations](https://mlsanity.com/paper/2308.14165/) (2023)
- [Recent Advances in the Foundations and Applications of Unbiased Learning to Rank](https://mlsanity.com/paper/2305.02914/) (2023)
- [A Control Function Framework for Mitigating Position Bias in Learning to Rank Systems](https://mlsanity.com/paper/2506.06989/) (2025)
- [Off-Policy Learning with Limited Supply](https://mlsanity.com/paper/2603.18702/) (2026)
- [Off-Policy Evaluation for Ranking Policies under Deterministic Logging Policies](https://mlsanity.com/paper/2603.21485/) (2026)
- [DOLCE: Decomposing Off-Policy Evaluation/Learning into Lagged and Current Effects](https://mlsanity.com/paper/2505.00961/) (2025)
- [Off-Policy Evaluation of Ranking Policies via Embedding-Space User Behavior Modeling](https://mlsanity.com/paper/2506.00446/) (2025)
- [Offline Contextual Bandits in the Presence of New Actions](https://mlsanity.com/paper/2605.18509/) (2026)
- [Credit-assigned Policy Gradient for Early Stage Retrieval in Two-stage Ranking](https://mlsanity.com/paper/2605.26385/) (2026)
- [Adaptive Doubly Robust Off-Policy Evaluation for Ranking Policies under Diverse User Behavior](https://mlsanity.com/paper/2608.29600/) (2026)
- [Recommendation Ranking Off-Policy Evaluation under Ranking-Dependent Examination via Examination-Relevance Decomposition](https://mlsanity.com/paper/2609.35034/) (2026)
- [Efficient Offline Learning of Ranking Policies via Top-k Policy Decomposition](https://mlsanity.com/paper/2609.36740/) (2026)

## Abstract (arXiv)

In real-world recommender systems and search engines, optimizing ranking decisions to present a ranked list of relevant items is critical. Off-policy evaluation (OPE) for ranking policies is thus gaining a growing interest because it enables performance estimation of new ranking policies using only logged data. Although OPE in contextual bandits has been studied extensively, its naive application to the ranking setting faces a critical variance issue due to the huge item space. To tackle this problem, previous studies introduce some assumptions on user behavior to make the combinatorial item space tractable. However, an unrealistic assumption may, in turn, cause serious bias. Therefore, appropriately controlling the bias-variance tradeoff by imposing a reasonable assumption is the key for success in OPE of ranking policies. To achieve a well-balanced bias-variance tradeoff, we propose the Cascade Doubly Robust estimator building on the cascade assumption, which assumes that a user interacts with items sequentially from the top position in a ranking. We show that the proposed estimator is unbiased in more cases compared to existing estimators that make stronger assumptions. Furthermore, compared to a previous estimator based on the same cascade assumption, the proposed estimator reduces the variance by leveraging a control variate. Comprehensive experiments on both synthetic and real-world data demonstrate that our estimator leads to more accurate OPE than existing estimators in a variety of settings.