mlsanityData to 14 Oct 2026

Adaptive Doubly Robust Off-Policy Evaluation for Ranking Policies under Diverse User Behavior

arXiv 2608.29600 ↗First posted 30 Aug 2026cs.LG cs.IRcited by 0

Not on the map. Why?

Abstract (arXiv)

Off-policy evaluation (OPE) of ranking policies is challenging be- cause selecting and ordering multiple items from a candidate set makes the number of possible rankings grow combinatorially with the number of candidates and the ranking length. Consequently, Inverse Propensity Scoring (IPS), whose importance weight is the full-ranking probability ratio under the evaluation and logging policies, can have excessive variance. Independent IPS (IIPS) and Reward Interaction IPS (RIPS) reduce variance by imposing fixed assumptions on how users browse rankings, but may introduce bias when those assumptions mismatch actual behavior. Adaptive Inverse Propensity Scoring (AIPS) addresses this trade-off by adap- tively marginalizing importance weights over the actions that affect each position-wise reward. It attains minimum variance within a class of unbiased IPS-based estimators when the true user be- havior model is observed. However, its estimation accuracy may still degrade for longer rankings, and AIPS does not use a reward model for residual correction. We propose Adaptive Doubly Robust (ADR), which combines adaptive importance weighting with re- ward regression through a control-variate correction. We establish its unbiasedness when the true user behavior model is observed and characterize a sufficient condition under which it reduces vari- ance relative to AIPS. Across synthetic experiments with 10,000 simulations per condition, ADR improves mean squared error over AIPS and conventional ranking OPE estimators across a range of logged-data sizes and ranking lengths.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 12 of its 19 references that are in the map. First posted on arXiv 30 Aug 2026.

Builds on

12 of 19 references.

References · 12

  1. 2016Optimal and Adaptive Off-policy Evaluation in Contextual Banditscited by 111
  2. 2016Off-policy evaluation for slate recommendationcited by 103
  3. 2018Offline A/B testing for Recommender Systemscited by 96
  4. 2019Doubly robust off-policy evaluation with shrinkagecited by 87
  5. 2022Off-Policy Evaluation for Large Action Spaces via Embeddingscited by 54
  6. 2020Counterfactual Evaluation of Slate Recommendations with Sequential Reward Interactionscited by 35
  7. 2023Off-Policy Evaluation for Large Action Spaces via Conjunct Effect Modelingcited by 34
  8. 2018Offline Evaluation of Ranking Policies with Click Modelscited by 32
  9. 2022Doubly Robust Off-Policy Evaluation for Ranking Policies under the Cascade Behavior Modelcited by 30
  10. 2024Off-Policy Evaluation of Slate Bandit Policies via Optimizing Abstractioncited by 17
  11. 2023Off-Policy Evaluation of Ranking Policies under Diverse User Behaviorcited by 16
  12. 2024Effective Off-Policy Evaluation and Learning in Contextual Combinatorial Banditscited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026From Weighting to Modeling: A Nonparametric Estimator for Off-Policy Evaluation
  2. 2024Towards Off-Policy Reinforcement Learning for Ranking Policies with Human Feedback
  3. 2021Off-Policy Evaluation Using Information Borrowing and Context-Based Switching
  4. 2023Learning Action Embeddings for Off-Policy Evaluation
  5. 2026Variance-Optimal Off-Policy Evaluation with Conjunct Effect Modeling
  6. 2026Efficient Offline Learning of Ranking Policies via Top-k Policy Decomposition
  7. 2023Unified Off-Policy Learning to Rank: a Reinforcement Learning Perspective
  8. 2025RewardRank: Optimizing True Learning-to-Rank Utility