# To Model or to Intervene: A Comparison of Counterfactual and Online Learning to Rank from User Interactions

arXiv: https://arxiv.org/abs/1907.06412 · HTML: https://mlsanity.com/paper/1907.06412/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jul 2019.

## Built on by (most cited first, 13 of 13)

- [Learning Robust Recommenders through Cross-Model Agreement](https://mlsanity.com/paper/2105.09605/) (2021)
- [RankFormer: Listwise Learning-to-Rank Using Listwide Labels](https://mlsanity.com/paper/2306.05808/) (2023)
- [Robust Generalization and Safe Query-Specialization in Counterfactual Learning to Rank](https://mlsanity.com/paper/2102.05990/) (2021)
- [BSL: Understanding and Improving Softmax Loss for Recommendation](https://mlsanity.com/paper/2312.12882/) (2023)
- [Intersection of Parallels as an Early Stopping Criterion](https://mlsanity.com/paper/2208.09529/) (2022)
- [Extending Open Bandit Pipeline to Simulate Industry Challenges](https://mlsanity.com/paper/2209.04147/) (2022)
- [Denoising Long- and Short-term Interests for Sequential Recommendation](https://mlsanity.com/paper/2407.14743/) (2024)
- [Learning Robust Recommender from Noisy Implicit Feedback](https://mlsanity.com/paper/2112.01160/) (2021)
- [Off-policy evaluation for learning-to-rank via interpolating the item-position model and the position-based model](https://mlsanity.com/paper/2210.09512/) (2022)
- [Unidentified and Confounded? Understanding Two-Tower Models for Unbiased Learning to Rank](https://mlsanity.com/paper/2506.20501/) (2025)
- [Label Denoising through Cross-Model Agreement](https://mlsanity.com/paper/2308.13976/) (2023)
- [SalesLoop: Reinforcement Learning from Performance Feedback for Sales Lead Ranking](https://mlsanity.com/paper/2607.20655/) (2026)
- [Embedding Surgery: Localized Updates for Adaptive Ranking Correction in Dense Retrieval](https://mlsanity.com/paper/2609.05110/) (2026)

## Abstract (arXiv)

Learning to Rank (LTR) from user interactions is challenging as user feedback often contains high levels of bias and noise. At the moment, two methodologies for dealing with bias prevail in the field of LTR: counterfactual methods that learn from historical data and model user behavior to deal with biases; and online methods that perform interventions to deal with bias but use no explicit user models. For practitioners the decision between either methodology is very important because of its direct impact on end users. Nevertheless, there has never been a direct comparison between these two approaches to unbiased LTR. In this study we provide the first benchmarking of both counterfactual and online LTR methods under different experimental conditions. Our results show that the choice between the methodologies is consequential and depends on the presence of selection bias, and the degree of position bias and interaction noise. In settings with little bias or noise counterfactual methods can obtain the highest ranking performance; however, in other circumstances their optimization can be detrimental to the user experience. Conversely, online methods are very robust to bias and noise but require control over the displayed rankings. Our findings confirm and contradict existing expectations on the impact of model-based and intervention-based methods in LTR, and allow practitioners to make an informed decision between the two methodologies.