# Differentiable Unbiased Online Learning to Rank

arXiv: https://arxiv.org/abs/1809.08415 · HTML: https://mlsanity.com/paper/1809.08415/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Sep 2018.

## Built on by (most cited first, 24 of 24)

- [Doubly-Robust Estimation for Correcting Position-Bias in Click Feedback for Unbiased Learning to Rank](https://mlsanity.com/paper/2203.17118/) (2022)
- [Robust Generalization and Safe Query-Specialization in Counterfactual Learning to Rank](https://mlsanity.com/paper/2102.05990/) (2021)
- [PairRank: Online Pairwise Learning to Rank by Divide-and-Conquer](https://mlsanity.com/paper/2103.00368/) (2021)
- [ULTRA: An Unbiased Learning To Rank Algorithm Toolbox](https://mlsanity.com/paper/2108.05073/) (2021)
- [Adversarial Attacks on Online Learning to Rank with Stochastic Click Models](https://mlsanity.com/paper/2305.19218/) (2023)
- [How to Forget Clients in Federated Online Learning to Rank?](https://mlsanity.com/paper/2401.13410/) (2024)
- [Do Lessons from Metric Learning Generalize to Image-Caption Retrieval?](https://mlsanity.com/paper/2202.07474/) (2022)
- [The Role of Relevance in Fair Ranking](https://mlsanity.com/paper/2305.05608/) (2023)
- [Inference-time Stochastic Ranking with Risk Control](https://mlsanity.com/paper/2306.07188/) (2023)
- [Unlearning for Federated Online Learning to Rank: A Reproducibility Study](https://mlsanity.com/paper/2505.12791/) (2025)
- [BayesCNS: A Unified Bayesian Approach to Address Cold Start and Non-Stationarity in Search Systems at Scale](https://mlsanity.com/paper/2410.02126/) (2024)
- [Calibrating Explore-Exploit Trade-off for Fair Online Learning to Rank](https://mlsanity.com/paper/2111.00735/) (2021)
- [Learning Neural Ranking Models Online from Implicit User Feedback](https://mlsanity.com/paper/2201.06658/) (2022)
- [Low-variance estimation in the Plackett-Luce model via quasi-Monte Carlo sampling](https://mlsanity.com/paper/2205.06024/) (2022)
- [Recent Advances in the Foundations and Applications of Unbiased Learning to Rank](https://mlsanity.com/paper/2305.02914/) (2023)
- [Optimizing Group-Fair Plackett-Luce Ranking Models for Relevance and Ex-Post Fairness](https://mlsanity.com/paper/2308.13242/) (2023)
- [RAIFLE: Reconstruction Attacks on Interaction-based Federated Learning with Adversarial Data Manipulation](https://mlsanity.com/paper/2310.19163/) (2023)
- [Investigating the Robustness of Counterfactual Learning to Rank Models: A Reproducibility Study](https://mlsanity.com/paper/2404.03707/) (2024)
- [A Large-Scale Web Search Dataset for Federated Online Learning to Rank](https://mlsanity.com/paper/2508.12353/) (2025)
- [Localization Boosting for Growth Markets: Mitigating Cross-Locale Behavioral Bias in Learning-to-Rank](https://mlsanity.com/paper/2605.11272/) (2026)
- [Exposure-Based Reinforcement Learning to Rank](https://mlsanity.com/paper/2607.18689/) (2026)
- [SalesLoop: Reinforcement Learning from Performance Feedback for Sales Lead Ranking](https://mlsanity.com/paper/2607.20655/) (2026)
- [Learning from What You Retrieve: Online RL Fine-Tuning for Semantic Retrieval](https://mlsanity.com/paper/2608.30753/) (2026)
- [Embedding Surgery: Localized Updates for Adaptive Ranking Correction in Dense Retrieval](https://mlsanity.com/paper/2609.05110/) (2026)

## Abstract (arXiv)

Online Learning to Rank (OLTR) methods optimize rankers based on user interactions. State-of-the-art OLTR methods are built specifically for linear models. Their approaches do not extend well to non-linear models such as neural networks. We introduce an entirely novel approach to OLTR that constructs a weighted differentiable pairwise loss after each interaction: Pairwise Differentiable Gradient Descent (PDGD). PDGD breaks away from the traditional approach that relies on interleaving or multileaving and extensive sampling of models to estimate gradients. Instead, its gradient is based on inferring preferences between document pairs from user clicks and can optimize any differentiable model. We prove that the gradient of PDGD is unbiased w.r.t. user document pair preferences. Our experiments on the largest publicly available Learning to Rank (LTR) datasets show considerable and significant improvements under all levels of interaction noise. PDGD outperforms existing OLTR methods both in terms of learning speed as well as final convergence. Furthermore, unlike previous OLTR methods, PDGD also allows for non-linear models to be optimized effectively. Our results show that using a neural network leads to even better performance at convergence than a linear model. In summary, PDGD is an efficient and unbiased OLTR approach that provides a better user experience than previously possible.