# Unidentified and Confounded? Understanding Two-Tower Models for Unbiased Learning to Rank

arXiv: https://arxiv.org/abs/2506.20501 · HTML: https://mlsanity.com/paper/2506.20501/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 59 references that are in the map.
- First posted on arXiv 25 Jun 2025.

## Builds on (most cited first, 17 of 17)

- [Revisiting Classifier Two-Sample Tests](https://mlsanity.com/paper/1610.06545/) (2016)
- [Generalized Sliced Wasserstein Distances](https://mlsanity.com/paper/1902.00434/) (2019)
- [Unbiased Learning-to-Rank with Biased Feedback](https://mlsanity.com/paper/1608.04468/) (2016)
- [Introducing LETOR 4.0 Datasets](https://mlsanity.com/paper/1306.2597/) (2013)
- [Unbiased Learning to Rank with Unbiased Propensity Estimation](https://mlsanity.com/paper/1804.05938/) (2018)
- [Beyond the Selected Completely At Random Assumption for Learning from Positive and Unlabeled Data](https://mlsanity.com/paper/1809.03207/) (2018)
- [Offline Evaluation of Ranking Policies with Click Models](https://mlsanity.com/paper/1804.10488/) (2018)
- [Estimating Position Bias without Intrusive Interventions](https://mlsanity.com/paper/1812.05161/) (2018)
- [Intervention Harvesting for Context-Dependent Examination-Bias Estimation](https://mlsanity.com/paper/1811.01802/) (2018)
- [To Model or to Intervene: A Comparison of Counterfactual and Online Learning to Rank from User Interactions](https://mlsanity.com/paper/1907.06412/) (2019)
- [A Large Scale Search Dataset for Unbiased Learning to Rank](https://mlsanity.com/paper/2207.03051/) (2022)
- [Reaching the End of Unbiasedness: Uncovering Implicit Limitations of Click-Based Learning to Rank](https://mlsanity.com/paper/2206.12204/) (2022)
- [Towards Disentangling Relevance and Bias in Unbiased Learning to Rank](https://mlsanity.com/paper/2212.13937/) (2022)
- [An Offline Metric for the Debiasedness of Click Models](https://mlsanity.com/paper/2304.09560/) (2023)
- [Multi-Feature Integration for Perception-Dependent Examination-Bias Estimation](https://mlsanity.com/paper/2302.13756/) (2023)
- [Identifiability Matters: Revealing the Hidden Recoverable Condition in Unbiased Learning to Rank](https://mlsanity.com/paper/2309.15560/) (2023)
- [Personalized Transformer-based Ranking for e-Commerce at Yandex](https://mlsanity.com/paper/2310.03481/) (2023)

## Built on by (most cited first, 1 of 1)

- [Recommendation Ranking Off-Policy Evaluation under Ranking-Dependent Examination via Examination-Relevance Decomposition](https://mlsanity.com/paper/2609.35034/) (2026)

## Abstract (arXiv)

Additive two-tower models are popular learning-to-rank methods for handling biased user feedback in industry settings. Recent studies, however, report a concerning phenomenon: training two-tower models on clicks collected by well-performing production systems leads to decreased ranking performance. This paper investigates two recent explanations for this observation: confounding effects from logging policies and model identifiability issues. We theoretically analyze the identifiability conditions of two-tower models, showing that either document swaps across positions or overlapping feature distributions are required to recover model parameters from clicks. We also investigate the effect of logging policies on two-tower models, finding that they introduce no bias when models perfectly capture user behavior. However, logging policies can amplify biases when models imperfectly capture user behavior, particularly when prediction errors correlate with document placement across positions. We propose a sample weighting technique to mitigate these effects and provide actionable insights for researchers and practitioners using two-tower models.