# Safe Deployment for Counterfactual Learning to Rank with Exposure-Based Risk Minimization

arXiv: https://arxiv.org/abs/2305.01522 · HTML: https://mlsanity.com/paper/2305.01522/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 53 references that are in the map.
- First posted on arXiv 26 Apr 2023.

## Builds on (most cited first, 13 of 13)

- [f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization](https://mlsanity.com/paper/1606.00709/) (2016)
- [Unbiased Learning-to-Rank with Biased Feedback](https://mlsanity.com/paper/1608.04468/) (2016)
- [Introducing LETOR 4.0 Datasets](https://mlsanity.com/paper/1306.2597/) (2013)
- [Open Bandit Dataset and Pipeline: Towards Realistic and Reproducible Off-Policy Evaluation](https://mlsanity.com/paper/2008.07146/) (2020)
- [Computationally Efficient Optimization of Plackett-Luce Ranking Models for Relevance and Fairness](https://mlsanity.com/paper/2105.00855/) (2021)
- [Policy-Gradient Training of Fair and Unbiased Ranking Functions](https://mlsanity.com/paper/1911.08054/) (2019)
- [Unifying Online and Counterfactual Learning to Rank](https://mlsanity.com/paper/2012.04426/) (2020)
- [Safe Exploration for Optimizing Contextual Bandits](https://mlsanity.com/paper/2002.00467/) (2020)
- [Reaching the End of Unbiasedness: Uncovering Implicit Limitations of Click-Based Learning to Rank](https://mlsanity.com/paper/2206.12204/) (2022)
- [Taking the Counterfactual Online: Efficient and Unbiased Online Evaluation for Ranking](https://mlsanity.com/paper/2007.12719/) (2020)
- [Robust Generalization and Safe Query-Specialization in Counterfactual Learning to Rank](https://mlsanity.com/paper/2102.05990/) (2021)
- [Learning from User Interactions with Rankings: A Unification of the Field](https://mlsanity.com/paper/2012.06576/) (2020)
- [Off-policy Learning for Multiple Loggers](https://mlsanity.com/paper/1907.09652/) (2019)

## Built on by (most cited first, 11 of 11)

- [A Simple and Effective Reinforcement Learning Method for Text-to-Image Diffusion Fine-tuning](https://mlsanity.com/paper/2503.00897/) (2025)
- [On (Normalised) Discounted Cumulative Gain as an Off-Policy Evaluation Metric for Top-n Recommendation](https://mlsanity.com/paper/2307.15053/) (2023)
- [Optimal Baseline Corrections for Off-Policy Contextual Bandits](https://mlsanity.com/paper/2405.05736/) (2024)
- [Unbiased Learning to Rank Meets Reality: Lessons from Baidu's Large-Scale Search Dataset](https://mlsanity.com/paper/2404.02543/) (2024)
- [Delta-rm OPE: Off-Policy Estimation with Pairs of Policies](https://mlsanity.com/paper/2405.10024/) (2024)
- [Practical and Robust Safety Guarantees for Advanced Counterfactual Learning to Rank](https://mlsanity.com/paper/2407.19943/) (2024)
- [Additive Control Variates Dominate Self-Normalisation in Off-Policy Evaluation](https://mlsanity.com/paper/2602.14914/) (2026)
- [Recent Advances in the Foundations and Applications of Unbiased Learning to Rank](https://mlsanity.com/paper/2305.02914/) (2023)
- [Investigating the Robustness of Counterfactual Learning to Rank Models: A Reproducibility Study](https://mlsanity.com/paper/2404.03707/) (2024)
- [A Simpler Alternative to Variational Regularized Counterfactual Risk Minimization](https://mlsanity.com/paper/2409.09819/) (2024)
- [Safe, Efficient, and Robust Reinforcement Learning for Ranking and Diffusion Models](https://mlsanity.com/paper/2510.15429/) (2025)

## Abstract (arXiv)

Counterfactual learning to rank (CLTR) relies on exposure-based inverse propensity scoring (IPS), a LTR-specific adaptation of IPS to correct for position bias. While IPS can provide unbiased and consistent estimates, it often suffers from high variance. Especially when little click data is available, this variance can cause CLTR to learn sub-optimal ranking behavior. Consequently, existing CLTR methods bring significant risks with them, as naively deploying their models can result in very negative user experiences. We introduce a novel risk-aware CLTR method with theoretical guarantees for safe deployment. We apply a novel exposure-based concept of risk regularization to IPS estimation for LTR. Our risk regularization penalizes the mismatch between the ranking behavior of a learned model and a given safe model. Thereby, it ensures that learned ranking models stay close to a trusted model, when there is high uncertainty in IPS estimation, which greatly reduces the risks during deployment. Our experimental results demonstrate the efficacy of our proposed method, which is effective at avoiding initial periods of bad performance when little data is available, while also maintaining high performance at convergence. For the CLTR field, our novel exposure-based risk minimization method enables practitioners to adopt CLTR methods in a safer manner that mitigates many of the risks attached to previous methods.