# Gender Stereotype Reinforcement: Measuring the Gender Bias Conveyed by Ranking Algorithms

arXiv: https://arxiv.org/abs/2009.01334 · HTML: https://mlsanity.com/paper/2009.01334/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Sep 2020.

## Built on by (most cited first, 9 of 9)

- [Societal Biases in Retrieved Contents: Measurement Framework and Adversarial Mitigation for BERT Rankers](https://mlsanity.com/paper/2104.13640/) (2021)
- [Fairness and Bias in Algorithmic Hiring: a Multidisciplinary Survey](https://mlsanity.com/paper/2309.13933/) (2023)
- [Measuring Fairness Under Unawareness of Sensitive Attributes: A Quantification-Based Approach](https://mlsanity.com/paper/2109.08549/) (2021)
- [Effective Controllable Bias Mitigation for Classification and Retrieval using Gate Adapters](https://mlsanity.com/paper/2401.16457/) (2024)
- [A Catalog of Fairness-Aware Practices in Machine Learning Engineering](https://mlsanity.com/paper/2408.16683/) (2024)
- [Break Out of a Pigeonhole: A Unified Framework for Examining Miscalibration, Bias, and Stereotype in Recommender Systems](https://mlsanity.com/paper/2312.17443/) (2023)
- [Towards Fair Rankings: Leveraging LLMs for Gender Bias Detection and Measurement](https://mlsanity.com/paper/2506.22372/) (2025)
- [Controlling Gender Bias in Retrieval via a Backpack Architecture](https://mlsanity.com/paper/2511.00875/) (2025)
- [WISE: Web Information Satire and Fakeness Evaluation](https://mlsanity.com/paper/2512.24000/) (2025)

## Abstract (arXiv)

Search Engines (SE) have been shown to perpetuate well-known gender stereotypes identified in psychology literature and to influence users accordingly. Similar biases were found encoded in Word Embeddings (WEs) learned from large online corpora. In this context, we propose the Gender Stereotype Reinforcement (GSR) measure, which quantifies the tendency of a SE to support gender stereotypes, leveraging gender-related information encoded in WEs. Through the critical lens of construct validity, we validate the proposed measure on synthetic and real collections. Subsequently, we use GSR to compare widely-used Information Retrieval ranking algorithms, including lexical, semantic, and neural models. We check if and how ranking algorithms based on WEs inherit the biases of the underlying embeddings. We also consider the most common debiasing approaches for WEs proposed in the literature and test their impact in terms of GSR and common performance measures. To the best of our knowledge, GSR is the first specifically tailored measure for IR, capable of quantifying representational harms.