# Measurement and applications of position bias in a marketplace search engine

arXiv: https://arxiv.org/abs/2206.11720 · HTML: https://mlsanity.com/paper/2206.11720/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 32 references that are in the map.
- First posted on arXiv 23 Jun 2022.

## Builds on (most cited first, 9 of 9)

- [Fairness of Exposure in Rankings](https://mlsanity.com/paper/1802.07281/) (2018)
- [Unbiased Learning-to-Rank with Biased Feedback](https://mlsanity.com/paper/1608.04468/) (2016)
- [Causal Intervention for Leveraging Popularity Bias in Recommendation](https://mlsanity.com/paper/2105.06067/) (2021)
- [Model-Agnostic Counterfactual Reasoning for Eliminating Popularity Bias in Recommender System](https://mlsanity.com/paper/2010.15363/) (2020)
- [Correcting for Selection Bias in Learning-to-rank Systems](https://mlsanity.com/paper/2001.11358/) (2020)
- [Estimating Position Bias without Intrusive Interventions](https://mlsanity.com/paper/1812.05161/) (2018)
- [Improving Deep Learning For Airbnb Search](https://mlsanity.com/paper/2002.05515/) (2020)
- [Handling Position Bias for Unbiased Learning to Rank in Hotels Search](https://mlsanity.com/paper/2002.12528/) (2020)
- [DebiasedRec: Bias-aware User Modeling and Click Prediction for Personalized News Recommendation](https://mlsanity.com/paper/2104.07360/) (2021)

## Built on by (most cited first, 2 of 2)

- [Position bias in features](https://mlsanity.com/paper/2402.02626/) (2024)
- [DCM Bandits: Multiplayer Information Asymmetric Cascading Bandits for Multiple Clicks](https://mlsanity.com/paper/2608.11873/) (2026)

## Abstract (arXiv)

Search engines intentionally influence user behavior by picking and ranking the list of results. Users engage with the highest results both because of their prominent placement and because they are typically the most relevant documents. Search engine ranking algorithms need to identify relevance while incorporating the influence of the search engine itself. This paper describes our efforts at Thumbtack to understand the impact of ranking, including the empirical results of a randomization program. In the context of a consumer marketplace we discuss practical details of model choice, experiment design, bias calculation, and machine learning model adaptation. We include a novel discussion of how ranking bias may not only affect labels, but also model features. The randomization program led to improved models, motivated internal scenario analysis, and enabled user-facing scenario tooling.