# Noise tolerance of learning to rank under class-conditional label noise

arXiv: https://arxiv.org/abs/2208.02126 · HTML: https://mlsanity.com/paper/2208.02126/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 18 references that are in the map.
- First posted on arXiv 3 Aug 2022.

## Builds on (most cited first, 6 of 6)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Distribution-Free, Risk-Controlling Prediction Sets](https://mlsanity.com/paper/2101.02703/) (2021)
- [Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Control](https://mlsanity.com/paper/2110.01052/) (2021)
- [A Theoretical Analysis of NDCG Type Ranking Measures](https://mlsanity.com/paper/1304.6480/) (2013)
- [The Landscape of Empirical Risk for Non-convex Losses](https://mlsanity.com/paper/1607.06534/) (2016)
- [Learning More From Less: Towards Strengthening Weak Supervision for Ad-Hoc Retrieval](https://mlsanity.com/paper/1907.08657/) (2019)

## Built on by (most cited first, 2 of 2)

- [ACES: Who Tests the Tests? Leave-One-Out AUC Consistency for Code Generation](https://mlsanity.com/paper/2604.03922/) (2026)
- [Nous: Learning and Certifying Memory Decisions Before Source Calibration](https://mlsanity.com/paper/2610.00094/) (2026)

## Abstract (arXiv)

Often, the data used to train ranking models is subject to label noise. For example, in web-search, labels created from clickstream data are noisy due to issues such as insufficient information in item descriptions on the SERP, query reformulation by the user, and erratic or unexpected user behavior. In practice, it is difficult to handle label noise without making strong assumptions about the label generation process. As a result, practitioners typically train their learning-to-rank (LtR) models directly on this noisy data without additional consideration of the label noise. Surprisingly, we often see strong performance from LtR models trained in this way. In this work, we describe a class of noise-tolerant LtR losses for which empirical risk minimization is a consistent procedure, even in the context of class-conditional label noise. We also develop noise-tolerant analogs of commonly used loss functions. The practical implications of our theoretical findings are further supported by experimental results.