# Effective Evaluation using Logged Bandit Feedback from Multiple Loggers

arXiv: https://arxiv.org/abs/1703.06180 · HTML: https://mlsanity.com/paper/1703.06180/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Mar 2017.

## Built on by (most cited first, 20 of 20)

- [A Review of Off-Policy Evaluation in Reinforcement Learning](https://mlsanity.com/paper/2212.06355/) (2022)
- [Off-Policy Evaluation via Adaptive Weighting with Data from Contextual Bandits](https://mlsanity.com/paper/2106.02029/) (2021)
- [Evaluating the Robustness of Off-Policy Evaluation](https://mlsanity.com/paper/2108.13703/) (2021)
- [Policy-Adaptive Estimator Selection for Off-Policy Evaluation](https://mlsanity.com/paper/2211.13904/) (2022)
- [On (Normalised) Discounted Cumulative Gain as an Off-Policy Evaluation Metric for Top-n Recommendation](https://mlsanity.com/paper/2307.15053/) (2023)
- [Long-term Off-Policy Evaluation and Learning](https://mlsanity.com/paper/2404.15691/) (2024)
- [Counterfactual-Augmented Importance Sampling for Semi-Offline Policy Evaluation](https://mlsanity.com/paper/2310.17146/) (2023)
- [Variance-Optimal Augmentation Logging for Counterfactual Evaluation in Contextual Bandits](https://mlsanity.com/paper/2202.01721/) (2022)
- [Simpson's Paradox in Recommender Fairness: Reconciling differences between per-user and aggregated evaluations](https://mlsanity.com/paper/2210.07755/) (2022)
- [Robust Offline Policy Learning with Observational Data from Multiple Sources](https://mlsanity.com/paper/2410.08537/) (2024)
- [Off-policy estimation with adaptively collected data: the power of online learning](https://mlsanity.com/paper/2411.12786/) (2024)
- [Exploiting Similarities in A/B Testing with Off-Policy Estimation](https://mlsanity.com/paper/2506.10677/) (2025)
- [Stable and Semi-stable Sampling Approaches for Continuously Used Samples](https://mlsanity.com/paper/2203.01381/) (2022)
- [Inverse Propensity Score based offline estimator for deterministic ranking lists using position bias](https://mlsanity.com/paper/2208.14980/) (2022)
- [Federated Offline Policy Learning](https://mlsanity.com/paper/2305.12407/) (2023)
- [Generalized Regret Analysis of Thompson Sampling using Fractional Posteriors](https://mlsanity.com/paper/2309.06349/) (2023)
- [Clustered KL-barycenter design for policy evaluation](https://mlsanity.com/paper/2503.02735/) (2025)
- [Counterfactual learning of new adaptive instructional policies using logged data](https://mlsanity.com/paper/2606.23015/) (2026)
- [A Stackelberg Framework for Resource-Aware LLM Agents: Learning, Repair, and Conditional Guarantees](https://mlsanity.com/paper/2606.23026/) (2026)
- [Cross-Domain Off-Policy Evaluation and Learning for Contextual Bandits](https://mlsanity.com/paper/2607.22012/) (2026)

## Abstract (arXiv)

Accurately evaluating new policies (e.g. ad-placement models, ranking functions, recommendation functions) is one of the key prerequisites for improving interactive systems. While the conventional approach to evaluation relies on online A/B tests, recent work has shown that counterfactual estimators can provide an inexpensive and fast alternative, since they can be applied offline using log data that was collected from a different policy fielded in the past. In this paper, we address the question of how to estimate the performance of a new target policy when we have log data from multiple historic policies. This question is of great relevance in practice, since policies get updated frequently in most online systems. We show that naively combining data from multiple logging policies can be highly suboptimal. In particular, we find that the standard Inverse Propensity Score (IPS) estimator suffers especially when logging and target policies diverge -- to a point where throwing away data improves the variance of the estimator. We therefore propose two alternative estimators which we characterize theoretically and compare experimentally. We find that the new estimators can provide substantially improved estimation accuracy.