Understanding the Effects of the Baidu-ULTR Logging Policy on Two-Tower Models
Not on the map. Why?
Abstract (arXiv)
Despite the popularity of the two-tower model for unbiased learning to rank (ULTR) tasks, recent work suggests that it suffers from a major limitation that could lead to its collapse in industry applications: the problem of logging policy confounding. Several potential solutions have even been proposed; however, the evaluation of these methods was mostly conducted using semi-synthetic simulation experiments. This paper bridges the gap between theory and practice by investigating the confounding problem on the largest real-world dataset, Baidu-ULTR. Our main contributions are threefold: 1) we show that the conditions for the confounding problem are given on Baidu-ULTR, 2) the confounding problem bears no significant effect on the two-tower model, and 3) we point to a potential mismatch between expert annotations, the golden standard in ULTR, and user click behavior.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 6 of its 23 references that are in the map. First posted on arXiv 18 Sep 2024.
Builds on
6 of 23 references.
References · 6
- 2018Learning a Deep Listwise Context Model for Ranking Refinementcited by 62
- 2018Unbiased Learning to Rank with Unbiased Propensity Estimationcited by 61
- 2022A Large Scale Search Dataset for Unbiased Learning to Rankcited by 13
- 2024Unbiased Learning to Rank Meets Reality: Lessons from Baidu's Large-Scale Search Datasetcited by 10
- 2022Towards Disentangling Relevance and Bias in Unbiased Learning to Rankcited by 9
- 2023Identifiability Matters: Revealing the Hidden Recoverable Condition in Unbiased Learning to Rankcited by 3
Built on by
Not cited yet.