# Quantifying Search Bias: Investigating Sources of Bias for Political Searches in Social Media

arXiv: https://arxiv.org/abs/1704.01347 · HTML: https://mlsanity.com/paper/1704.01347/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Apr 2017.

## Built on by (most cited first, 18 of 18)

- [Societal Biases in Retrieved Contents: Measurement Framework and Adversarial Mitigation for BERT Rankers](https://mlsanity.com/paper/2104.13640/) (2021)
- [When Fair Ranking Meets Uncertain Inference](https://mlsanity.com/paper/2105.02091/) (2021)
- [Problematic Machine Behavior: A Systematic Literature Review of Algorithm Audits](https://mlsanity.com/paper/2102.04256/) (2021)
- [AI Can Be Cognitively Biased: An Exploratory Study on Threshold Priming in LLM-Based Batch Relevance Assessment](https://mlsanity.com/paper/2409.16022/) (2024)
- [Unmasking Nationality Bias: A Study of Human Perception of Nationalities in AI-Generated Articles](https://mlsanity.com/paper/2308.04346/) (2023)
- [Invisible Relevance Bias: Text-Image Retrieval Models Prefer AI-Generated Images](https://mlsanity.com/paper/2311.14084/) (2023)
- [Studying Up Machine Learning Data: Why Talk About Bias When We Mean Power?](https://mlsanity.com/paper/2109.08131/) (2021)
- [Measuring Disparate Outcomes of Content Recommendation Algorithms with Distributional Inequality Metrics](https://mlsanity.com/paper/2202.01615/) (2022)
- [Fine-Grained Prediction of Political Leaning on Social Media with Unsupervised Deep Learning](https://mlsanity.com/paper/2202.12382/) (2022)
- [Effective Controllable Bias Mitigation for Classification and Retrieval using Gate Adapters](https://mlsanity.com/paper/2401.16457/) (2024)
- [Measuring and Addressing Indexical Bias in Information Retrieval](https://mlsanity.com/paper/2406.04298/) (2024)
- [Mitigating Bias in RAG: Controlling the Embedder](https://mlsanity.com/paper/2502.17390/) (2025)
- [I Am Not Them: Fluid Identities and Persistent Out-group Bias in Large Language Models](https://mlsanity.com/paper/2402.10436/) (2024)
- [The End of Rented Discovery: How AI Search Redistributes Power Between Hotels and Intermediaries](https://mlsanity.com/paper/2603.20062/) (2026)
- [Fairness in Image Search: A Study of Occupational Stereotyping in Image Retrieval and its Debiasing](https://mlsanity.com/paper/2305.03881/) (2023)
- [Fuck the Algorithm: Conceptual Issues in Algorithmic Bias](https://mlsanity.com/paper/2505.13509/) (2025)
- [Learning to Rank with Top-K Fairness](https://mlsanity.com/paper/2509.18067/) (2025)
- [On Epistemic Diversity in Large Language Models](https://mlsanity.com/paper/2609.04835/) (2026)

## Abstract (arXiv)

Search systems in online social media sites are frequently used to find information about ongoing events and people. For topics with multiple competing perspectives, such as political events or political candidates, bias in the top ranked results significantly shapes public opinion. However, bias does not emerge from an algorithm alone. It is important to distinguish between the bias that arises from the data that serves as the input to the ranking system and the bias that arises from the ranking system itself. In this paper, we propose a framework to quantify these distinct biases and apply this framework to politics-related queries on Twitter. We found that both the input data and the ranking system contribute significantly to produce varying amounts of bias in the search results and in different ways. We discuss the consequences of these biases and possible mechanisms to signal this bias in social media search systems' interfaces.