# Measuring Personalization of Web Search

arXiv: https://arxiv.org/abs/1706.05011 · HTML: https://mlsanity.com/paper/1706.05011/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jun 2017.

## Built on by (most cited first, 20 of 20)

- [Outlining Traceability: A Principle for Operationalizing Accountability in Computing Systems](https://mlsanity.com/paper/2101.09385/) (2021)
- [Problematic Machine Behavior: A Systematic Literature Review of Algorithm Audits](https://mlsanity.com/paper/2102.04256/) (2021)
- [An Audit of Misinformation Filter Bubbles on YouTube: Bubble Bursting and Recent Behavior Changes](https://mlsanity.com/paper/2203.13769/) (2022)
- [From Transparency to Accountability and Back: A Discussion of Access and Evidence in AI Auditing](https://mlsanity.com/paper/2410.04772/) (2024)
- [T-RECS: A Simulation Tool to Study the Societal Impact of Recommender Systems](https://mlsanity.com/paper/2107.08959/) (2021)
- [A Survey on Echo Chambers on Social Media: Description, Detection and Mitigation](https://mlsanity.com/paper/2112.05084/) (2021)
- [Auditing YouTube's Recommendation Algorithm for Misinformation Filter Bubbles](https://mlsanity.com/paper/2210.10085/) (2022)
- [Source Coverage and Citation Bias in LLM-based vs. Traditional Search Engines](https://mlsanity.com/paper/2512.09483/) (2025)
- [Using LLMs in Generating Design Rationale for Software Architecture Decisions](https://mlsanity.com/paper/2504.20781/) (2025)
- [PerSEval: Assessing Personalization in Text Summarizers](https://mlsanity.com/paper/2407.00453/) (2024)
- [Privacy-Utility-Bias Trade-offs for Privacy-Preserving Recommender Systems](https://mlsanity.com/paper/2511.22515/) (2025)
- [How Generative AI Disrupts Search: An Empirical Study of Google Search, Gemini, and AI Overviews](https://mlsanity.com/paper/2604.27790/) (2026)
- [Digital Gatekeepers: Google's Role in Curating Hashtags and Subreddits](https://mlsanity.com/paper/2506.14370/) (2025)
- [Improving User Privacy in Personalized Generation: Client-Side Retrieval-Augmented Modification of Server-Side Generated Speculations](https://mlsanity.com/paper/2601.17569/) (2026)
- [Measuring Google AI Overviews: Activation, Source Quality, Claim Fidelity, and Publisher Impact](https://mlsanity.com/paper/2605.14021/) (2026)
- [The Language of the Question Selects the Market: Query Language and Exit IP as Separable Factors in Commercial Recommendations from a Generative Search Interface](https://mlsanity.com/paper/2608.30052/) (2026)
- [Adapting Dual-encoder Vision-language Models for Paraphrased Retrieval](https://mlsanity.com/paper/2405.03190/) (2024)
- [Auditing Google's Search Algorithm: Measuring News Diversity Across Brazil, the UK, and the US](https://mlsanity.com/paper/2410.23842/) (2024)
- [TourMart: A Parametric Audit Instrument for Commission Steering in LLM Travel Agents](https://mlsanity.com/paper/2605.10440/) (2026)
- [The Mutations of Machine Speech](https://mlsanity.com/paper/2609.09496/) (2026)

## Abstract (arXiv)

Web search is an integral part of our daily lives. Recently, there has been a trend of personalization in Web search, where different users receive different results for the same search query. The increasing level of personalization is leading to concerns about Filter Bubble effects, where certain users are simply unable to access information that the search engines' algorithm decides is irrelevant. Despite these concerns, there has been little quantification of the extent of personalization in Web search today, or the user attributes that cause it. In light of this situation, we make three contributions. First, we develop a methodology for measuring personalization in Web search results. While conceptually simple, there are numerous details that our methodology must handle in order to accurately attribute differences in search results to personalization. Second, we apply our methodology to 200 users on Google Web Search and 100 users on Bing. We find that, on average, 11.7% of results show differences due to personalization on Google, while 15.8% of results are personalized on Bing, but that this varies widely by search query and by result ranking. Third, we investigate the user features used to personalize on Google Web Search and Bing. Surprisingly, we only find measurable personalization as a result of searching with a logged in account and the IP address of the searching user. Our results are a first step towards understanding the extent and effects of personalization on Web search engines today.