mlsanityData to 7 Oct 2026

Do Neural Ranking Models Intensify Gender Bias?

arXiv 2005.00372 ↗First posted 1 May 2020cs.IR cs.LGcited by 28

Not on the map. Why?

Abstract (arXiv)

Concerns regarding the footprint of societal biases in information retrieval (IR) systems have been raised in several previous studies. In this work, we examine various recent IR models from the perspective of the degree of gender bias in their retrieval results. To this end, we first provide a bias measurement framework which includes two metrics to quantify the degree of the unbalanced presence of gender-related concepts in a given IR model's ranking list. To examine IR models by means of the framework, we create a dataset of non-gendered queries, selected by human annotators. Applying these queries to the MS MARCO Passage retrieval collection, we then measure the gender bias of a BM25 model and several recent neural ranking models. The results show that while all models are strongly biased toward male, the neural models, and in particular the ones based on contextualized embedding models, significantly intensify gender bias. Our experiments also show an overall increase in the gender bias of neural models when they exploit transfer learning, namely when they use (already biased) pre-trained embeddings.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 5; 2023: 4; 2024: 7; 2025: 6; 2026: 2.

28 papers.

Sort:

By area:

  1. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  2. 2021WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language modelscited by 89
  3. 2021Societal Biases in Retrieved Contents: Measurement Framework and Adversarial Mitigation for BERT Rankerscited by 34
  4. 2025Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Surveycited by 29
  5. 2024Does RAG Introduce Unfairness in LLMs? Evaluating Fairness in Retrieval-Augmented Generation Systemscited by 26
  6. 2023Parameter-efficient Modularised Bias Mitigation via AdapterFusioncited by 25
  7. 2024Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas: A Surveycited by 23
  8. 2022Unlearning Protected User Attributes in Recommendations with Adversarial Trainingcited by 19
  9. 2022Modular and On-demand Bias Mitigation with Attribute-Removal Subnetworkscited by 18
  10. 2021Mitigating the Position Bias of Transformer Models in Passage Re-Rankingcited by 10
  11. 2024Effective Controllable Bias Mitigation for Classification and Retrieval using Gate Adapterscited by 8
  12. 2022HumSet: Dataset of Multilingual Information Extraction and Classification for Humanitarian Crisis Responsecited by 7
16 more
  1. 2025Evaluating the Effect of Retrieval Augmentation on Social Biasescited by 7
  2. 2024Measuring and Addressing Indexical Bias in Information Retrievalcited by 6
  3. 2025Mitigating Bias in RAG: Controlling the Embeddercited by 6
  4. 2022Debiasing Neural Retrieval via In-batch Balancing Regularizationcited by 5
  5. 2021CODER: An efficient framework for improving retrieval through COntextual Document Embedding Rerankingcited by 4
  6. 2024Measuring Bias in a Ranked List using Term-based Representationscited by 4
  7. 2022Debiasing Gender Bias in Information Retrieval Modelscited by 2
  8. 2023Leveraging Domain Knowledge for Inclusive and Bias-aware Humanitarian Response Entry Classificationcited by 2
  9. 2023EALM: Introducing Multidimensional Ethical Alignment in Conversational Information Retrievalcited by 2
  10. 2025Investigating Gender Bias in LLM-Generated Stories via Psychological Stereotypescited by 2
  11. 2024MultiContrievers: Analysis of Dense Retrieval Representationscited by 1
  12. 2026With Argus Eyes: Assessing Retrieval Gaps via Uncertainty Scoring to Detect and Remedy Retrieval Blind Spotscited by 1
  13. 2026Social Bias in LLM-Generated Code: Benchmark and Mitigationcited by 1
  14. 2024Revisiting The Classics: A Study on Identifying and Rectifying Gender Stereotypes in Rhymes and Poemscited by 0
  15. 2025Towards Fair Rankings: Leveraging LLMs for Gender Bias Detection and Measurementcited by 0
  16. 2025Controlling Gender Bias in Retrieval via a Backpack Architecturecited by 0