# Societal Biases in Retrieved Contents: Measurement Framework and Adversarial Mitigation for BERT Rankers

arXiv: https://arxiv.org/abs/2104.13640 · HTML: https://mlsanity.com/paper/2104.13640/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 63 references that are in the map.
- First posted on arXiv 28 Apr 2021.

## Builds on (most cited first, 30 of 36)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT](https://mlsanity.com/paper/2004.12832/) (2020)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval](https://mlsanity.com/paper/2007.00808/) (2020)
- [Mitigating Unwanted Biases with Adversarial Learning](https://mlsanity.com/paper/1801.07593/) (2018)
- [Passage Re-ranking with BERT](https://mlsanity.com/paper/1901.04085/) (2019)
- [Learning Adversarially Fair and Transferable Representations](https://mlsanity.com/paper/1802.06309/) (2018)
- [Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting](https://mlsanity.com/paper/1901.09451/) (2019)
- [Overview of the TREC 2019 deep learning track](https://mlsanity.com/paper/2003.07820/) (2020)
- [Fairness of Exposure in Rankings](https://mlsanity.com/paper/1802.07281/) (2018)
- [Adversarial Removal of Demographic Attributes from Text Data](https://mlsanity.com/paper/1808.06640/) (2018)
- [Fairness-Aware Ranking in Search & Recommendation Systems with Application to LinkedIn Talent Search](https://mlsanity.com/paper/1905.01989/) (2019)
- [Controllable Invariance through Adversarial Feature Learning](https://mlsanity.com/paper/1705.11122/) (2017)
- [Equity of Attention: Amortizing Individual Fairness in Rankings](https://mlsanity.com/paper/1805.01788/) (2018)
- [FA*IR: A Fair Top-k Ranking Algorithm](https://mlsanity.com/paper/1706.06368/) (2017)
- [It's All in the Name: Mitigating Gender Bias with Name-Based Counterfactual Data Substitution](https://mlsanity.com/paper/1909.00871/) (2019)
- [End-to-End Neural Ad-hoc Ranking with Kernel Pooling](https://mlsanity.com/paper/1706.06613/) (2017)
- [Measuring Fairness in Ranked Outputs](https://mlsanity.com/paper/1610.08559/) (2016)
- [Ranking with Fairness Constraints](https://mlsanity.com/paper/1704.06840/) (2017)
- [Policy Learning for Fairness in Ranking](https://mlsanity.com/paper/1902.04056/) (2019)
- [Controlling Fairness and Bias in Dynamic Learning-to-Rank](https://mlsanity.com/paper/2005.14713/) (2020)
- [Text Matching as Image Recognition](https://mlsanity.com/paper/1602.06359/) (2016)
- [What's in a Name? Reducing Bias in Bios without Access to Protected Attributes](https://mlsanity.com/paper/1904.05233/) (2019)
- [Efficient Document Re-Ranking for Transformers by Precomputing Term Representations](https://mlsanity.com/paper/2004.14255/) (2020)
- [Reducing Disparate Exposure in Ranking: A Learning To Rank Approach](https://mlsanity.com/paper/1805.08716/) (2018)

## Built on by (most cited first, 30 of 35)

- [Bias and Fairness in Large Language Models: A Survey](https://mlsanity.com/paper/2309.00770/) (2023)
- [WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models](https://mlsanity.com/paper/2112.06598/) (2021)
- [Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2502.06872/) (2025)
- [Does RAG Introduce Unfairness in LLMs? Evaluating Fairness in Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2409.19804/) (2024)
- [Parameter-efficient Modularised Bias Mitigation via AdapterFusion](https://mlsanity.com/paper/2302.06321/) (2023)
- [Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas: A Survey](https://mlsanity.com/paper/2406.05392/) (2024)
- [Joint Multisided Exposure Fairness for Recommendation](https://mlsanity.com/paper/2205.00048/) (2022)
- [Debiasing Methods for Fairer Neural Models in Vision and Language Research: A Survey](https://mlsanity.com/paper/2211.05617/) (2022)
- [Unlearning Protected User Attributes in Recommendations with Adversarial Training](https://mlsanity.com/paper/2206.04500/) (2022)
- [Modular and On-demand Bias Mitigation with Attribute-Removal Subnetworks](https://mlsanity.com/paper/2205.15171/) (2022)
- [Tackling Bias in Pre-trained Language Models: Current Trends and Under-represented Societies](https://mlsanity.com/paper/2312.01509/) (2023)
- [A Multidimensional Analysis of Social Biases in Vision Transformers](https://mlsanity.com/paper/2308.01948/) (2023)
- [Effective Controllable Bias Mitigation for Classification and Retrieval using Gate Adapters](https://mlsanity.com/paper/2401.16457/) (2024)
- [HumSet: Dataset of Multilingual Information Extraction and Classification for Humanitarian Crisis Response](https://mlsanity.com/paper/2210.04573/) (2022)
- [Measuring and Addressing Indexical Bias in Information Retrieval](https://mlsanity.com/paper/2406.04298/) (2024)
- [Debiasing Neural Retrieval via In-batch Balancing Regularization](https://mlsanity.com/paper/2205.09240/) (2022)
- [Bias-Aware Agent: Enhancing Fairness in AI-Driven Knowledge Retrieval](https://mlsanity.com/paper/2503.21237/) (2025)
- [Political Ideology Shifts in Large Language Models](https://mlsanity.com/paper/2508.16013/) (2025)
- [CODER: An efficient framework for improving retrieval through COntextual Document Embedding Reranking](https://mlsanity.com/paper/2112.08766/) (2021)
- [Measuring Bias in a Ranked List using Term-based Representations](https://mlsanity.com/paper/2403.05975/) (2024)
- [Simulating a Bias Mitigation Scenario in Large Language Models](https://mlsanity.com/paper/2509.14438/) (2025)
- [Metricizing the Euclidean Space towards Desired Distance Relations in Point Clouds](https://mlsanity.com/paper/2211.03674/) (2022)
- [Unlabeled Debiasing in Downstream Tasks via Class-wise Low Variance Regularization](https://mlsanity.com/paper/2409.19541/) (2024)
- [Masked Audio Text Encoders are Effective Multi-Modal Rescorers](https://mlsanity.com/paper/2305.07677/) (2023)
- [Leveraging Domain Knowledge for Inclusive and Bias-aware Humanitarian Response Entry Classification](https://mlsanity.com/paper/2305.16756/) (2023)
- [Towards Better Fairness-Utility Trade-off: A Comprehensive Measurement-Based Reinforcement Learning Framework](https://mlsanity.com/paper/2307.11379/) (2023)
- [Perception of Visual Content: Differences Between Humans and Foundation Models](https://mlsanity.com/paper/2411.18968/) (2024)
- [Bias Mitigation Agent: Optimizing Source Selection for Fair and Balanced Knowledge Retrieval](https://mlsanity.com/paper/2508.18724/) (2025)
- [Ideology-Based LLMs for Content Moderation](https://mlsanity.com/paper/2510.25805/) (2025)
- [With Argus Eyes: Assessing Retrieval Gaps via Uncertainty Scoring to Detect and Remedy Retrieval Blind Spots](https://mlsanity.com/paper/2602.09616/) (2026)

## Abstract (arXiv)

Societal biases resonate in the retrieved contents of information retrieval (IR) systems, resulting in reinforcing existing stereotypes. Approaching this issue requires established measures of fairness in respect to the representation of various social groups in retrieval results, as well as methods to mitigate such biases, particularly in the light of the advances in deep ranking models. In this work, we first provide a novel framework to measure the fairness in the retrieved text contents of ranking models. Introducing a ranker-agnostic measurement, the framework also enables the disentanglement of the effect on fairness of collection from that of rankers. To mitigate these biases, we propose AdvBert, a ranking model achieved by adapting adversarial bias mitigation for IR, which jointly learns to predict relevance and remove protected attributes. We conduct experiments on two passage retrieval collections (MSMARCO Passage Re-ranking and TREC Deep Learning 2019 Passage Re-ranking), which we extend by fairness annotations of a selected subset of queries regarding gender attributes. Our results on the MSMARCO benchmark show that, (1) all ranking models are less fair in comparison with ranker-agnostic baselines, and (2) the fairness of Bert rankers significantly improves when using the proposed AdvBert models. Lastly, we investigate the trade-off between fairness and utility, showing that we can maintain the significant improvements in fairness without any significant loss in utility.