# Do Neural Ranking Models Intensify Gender Bias?

arXiv: https://arxiv.org/abs/2005.00372 · HTML: https://mlsanity.com/paper/2005.00372/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 May 2020.

## Built on by (most cited first, 29 of 29)

- [Bias and Fairness in Large Language Models: A Survey](https://mlsanity.com/paper/2309.00770/) (2023)
- [WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models](https://mlsanity.com/paper/2112.06598/) (2021)
- [Societal Biases in Retrieved Contents: Measurement Framework and Adversarial Mitigation for BERT Rankers](https://mlsanity.com/paper/2104.13640/) (2021)
- [Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2502.06872/) (2025)
- [Does RAG Introduce Unfairness in LLMs? Evaluating Fairness in Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2409.19804/) (2024)
- [Parameter-efficient Modularised Bias Mitigation via AdapterFusion](https://mlsanity.com/paper/2302.06321/) (2023)
- [Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas: A Survey](https://mlsanity.com/paper/2406.05392/) (2024)
- [Unlearning Protected User Attributes in Recommendations with Adversarial Training](https://mlsanity.com/paper/2206.04500/) (2022)
- [Modular and On-demand Bias Mitigation with Attribute-Removal Subnetworks](https://mlsanity.com/paper/2205.15171/) (2022)
- [Mitigating the Position Bias of Transformer Models in Passage Re-Ranking](https://mlsanity.com/paper/2101.06980/) (2021)
- [Effective Controllable Bias Mitigation for Classification and Retrieval using Gate Adapters](https://mlsanity.com/paper/2401.16457/) (2024)
- [HumSet: Dataset of Multilingual Information Extraction and Classification for Humanitarian Crisis Response](https://mlsanity.com/paper/2210.04573/) (2022)
- [Evaluating the Effect of Retrieval Augmentation on Social Biases](https://mlsanity.com/paper/2502.17611/) (2025)
- [Measuring and Addressing Indexical Bias in Information Retrieval](https://mlsanity.com/paper/2406.04298/) (2024)
- [Mitigating Bias in RAG: Controlling the Embedder](https://mlsanity.com/paper/2502.17390/) (2025)
- [Debiasing Neural Retrieval via In-batch Balancing Regularization](https://mlsanity.com/paper/2205.09240/) (2022)
- [CODER: An efficient framework for improving retrieval through COntextual Document Embedding Reranking](https://mlsanity.com/paper/2112.08766/) (2021)
- [Measuring Bias in a Ranked List using Term-based Representations](https://mlsanity.com/paper/2403.05975/) (2024)
- [Debiasing Gender Bias in Information Retrieval Models](https://mlsanity.com/paper/2208.01755/) (2022)
- [Leveraging Domain Knowledge for Inclusive and Bias-aware Humanitarian Response Entry Classification](https://mlsanity.com/paper/2305.16756/) (2023)
- [EALM: Introducing Multidimensional Ethical Alignment in Conversational Information Retrieval](https://mlsanity.com/paper/2310.00970/) (2023)
- [Investigating Gender Bias in LLM-Generated Stories via Psychological Stereotypes](https://mlsanity.com/paper/2508.03292/) (2025)
- [MultiContrievers: Analysis of Dense Retrieval Representations](https://mlsanity.com/paper/2402.15925/) (2024)
- [With Argus Eyes: Assessing Retrieval Gaps via Uncertainty Scoring to Detect and Remedy Retrieval Blind Spots](https://mlsanity.com/paper/2602.09616/) (2026)
- [Social Bias in LLM-Generated Code: Benchmark and Mitigation](https://mlsanity.com/paper/2605.00382/) (2026)
- [Revisiting The Classics: A Study on Identifying and Rectifying Gender Stereotypes in Rhymes and Poems](https://mlsanity.com/paper/2403.11752/) (2024)
- [Towards Fair Rankings: Leveraging LLMs for Gender Bias Detection and Measurement](https://mlsanity.com/paper/2506.22372/) (2025)
- [Controlling Gender Bias in Retrieval via a Backpack Architecture](https://mlsanity.com/paper/2511.00875/) (2025)
- [Bias Robustness of Genetic Multi-Criteria Sorting versus Learned Rankers under Proxy Skew and Label Contamination](https://mlsanity.com/paper/2610.14015/) (2026)

## Abstract (arXiv)

Concerns regarding the footprint of societal biases in information retrieval (IR) systems have been raised in several previous studies. In this work, we examine various recent IR models from the perspective of the degree of gender bias in their retrieval results. To this end, we first provide a bias measurement framework which includes two metrics to quantify the degree of the unbalanced presence of gender-related concepts in a given IR model's ranking list. To examine IR models by means of the framework, we create a dataset of non-gendered queries, selected by human annotators. Applying these queries to the MS MARCO Passage retrieval collection, we then measure the gender bias of a BM25 model and several recent neural ranking models. The results show that while all models are strongly biased toward male, the neural models, and in particular the ones based on contextualized embedding models, significantly intensify gender bias. Our experiments also show an overall increase in the gender bias of neural models when they exploit transfer learning, namely when they use (already biased) pre-trained embeddings.