# Online Learning to Rank in Stochastic Click Models

arXiv: https://arxiv.org/abs/1703.02527 · HTML: https://mlsanity.com/paper/1703.02527/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Mar 2017.

## Built on by (most cited first, 26 of 26)

- [Optimal Design for Human Preference Elicitation](https://mlsanity.com/paper/2404.13895/) (2024)
- [Sample Efficient Demonstration Selection for In-Context Learning](https://mlsanity.com/paper/2506.08607/) (2025)
- [Minimax Regret for Cascading Bandits](https://mlsanity.com/paper/2203.12577/) (2022)
- [PairRank: Online Pairwise Learning to Rank by Divide-and-Conquer](https://mlsanity.com/paper/2103.00368/) (2021)
- [Adversarial Attacks on Online Learning to Rank with Click Feedback](https://mlsanity.com/paper/2305.17071/) (2023)
- [Adversarial Attacks on Online Learning to Rank with Stochastic Click Models](https://mlsanity.com/paper/2305.19218/) (2023)
- [Pure Exploration and Regret Minimization in Matching Bandits](https://mlsanity.com/paper/2108.00230/) (2021)
- [Unified Off-Policy Learning to Rank: a Reinforcement Learning Perspective](https://mlsanity.com/paper/2306.07528/) (2023)
- [Cascading Reinforcement Learning](https://mlsanity.com/paper/2401.08961/) (2024)
- [Simultaneously Learning Stochastic and Adversarial Bandits under the Position-Based Model](https://mlsanity.com/paper/2207.05437/) (2022)
- [Bandit Learning to Rank with Position-Based Click Models: Personalized and Equal Treatments](https://mlsanity.com/paper/2311.04528/) (2023)
- [BayesCNS: A Unified Bayesian Approach to Address Cold Start and Non-Stationarity in Search Systems at Scale](https://mlsanity.com/paper/2410.02126/) (2024)
- [CORe: Capitalizing On Rewards in Bandit Exploration](https://mlsanity.com/paper/2103.04387/) (2021)
- [Calibrating Explore-Exploit Trade-off for Fair Online Learning to Rank](https://mlsanity.com/paper/2111.00735/) (2021)
- [Learning Neural Ranking Models Online from Implicit User Feedback](https://mlsanity.com/paper/2201.06658/) (2022)
- [Targeted Advertising on Social Networks Using Online Variational Tensor Regression](https://mlsanity.com/paper/2208.10627/) (2022)
- [Overcoming Prior Misspecification in Online Learning to Rank](https://mlsanity.com/paper/2301.10651/) (2023)
- [Misalignment, Learning, and Ranking: Harnessing Users Limited Attention](https://mlsanity.com/paper/2402.14013/) (2024)
- [Evaluation of Agents under Simulated AI Marketplace Dynamics](https://mlsanity.com/paper/2604.14256/) (2026)
- [On Learning to Rank Long Sequences with Contextual Bandits](https://mlsanity.com/paper/2106.03546/) (2021)
- [UniRank: Unimodal Bandit Algorithm for Online Ranking](https://mlsanity.com/paper/2208.01515/) (2022)
- [Exploration of Unranked Items in Safe Online Learning to Re-Rank](https://mlsanity.com/paper/2305.01202/) (2023)
- [Ranking with Popularity Bias: User Welfare under Self-Amplification Dynamics](https://mlsanity.com/paper/2305.18333/) (2023)
- [Observation-Free Attacks on Online Learning to Rank](https://mlsanity.com/paper/2509.22855/) (2025)
- [Online Learning to Rank under Corruption: A Robust Cascading Bandits Approach](https://mlsanity.com/paper/2511.03074/) (2025)
- [Data Efficient Sample Selection for In-Context Learning](https://mlsanity.com/paper/2609.06670/) (2026)

## Abstract (arXiv)

Online learning to rank is a core problem in information retrieval and machine learning. Many provably efficient algorithms have been recently proposed for this problem in specific click models. The click model is a model of how the user interacts with a list of documents. Though these results are significant, their impact on practice is limited, because all proposed algorithms are designed for specific click models and lack convergence guarantees in other models. In this work, we propose BatchRank, the first online learning to rank algorithm for a broad class of click models. The class encompasses two most fundamental click models, the cascade and position-based models. We derive a gap-dependent upper bound on the T-step regret of BatchRank and evaluate it on a range of web search queries. We observe that BatchRank outperforms ranked bandits and is more robust than CascadeKL-UCB, an existing algorithm for the cascade model.