# A Survey on Gender Bias in Natural Language Processing

arXiv: https://arxiv.org/abs/2112.14168 · HTML: https://mlsanity.com/paper/2112.14168/

## Key facts

- Cited by 89 papers in the map (data through 14 Oct 2026).
- Builds on 84 of its 169 references that are in the map.
- First posted on arXiv 28 Dec 2021.

## Builds on (most cited first, 30 of 84)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [StereoSet: Measuring stereotypical bias in pretrained language models](https://mlsanity.com/paper/2004.09456/) (2020)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models](https://mlsanity.com/paper/2010.00133/) (2020)
- [Mitigating Unwanted Biases with Adversarial Learning](https://mlsanity.com/paper/1801.07593/) (2018)
- [Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods](https://mlsanity.com/paper/1804.06876/) (2018)
- [Gender Bias in Coreference Resolution](https://mlsanity.com/paper/1804.09301/) (2018)
- [Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints](https://mlsanity.com/paper/1707.09457/) (2017)
- [On Measuring Social Biases in Sentence Encoders](https://mlsanity.com/paper/1903.10561/) (2019)
- [A Multiscale Visualization of Attention in the Transformer Model](https://mlsanity.com/paper/1906.05714/) (2019)
- [Mitigating Gender Bias in Natural Language Processing: Literature Review](https://mlsanity.com/paper/1906.08976/) (2019)
- [Measuring Bias in Contextualized Word Representations](https://mlsanity.com/paper/1906.07337/) (2019)
- [Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them](https://mlsanity.com/paper/1903.03862/) (2019)
- [Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes](https://mlsanity.com/paper/1711.08412/) (2017)
- [Evaluating Gender Bias in Machine Translation](https://mlsanity.com/paper/1906.00591/) (2019)
- [Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems](https://mlsanity.com/paper/1805.04508/) (2018)
- [Identifying and Reducing Gender Bias in Word-Level Language Models](https://mlsanity.com/paper/1904.03035/) (2019)
- [Gender Bias in Neural Natural Language Processing](https://mlsanity.com/paper/1807.11714/) (2018)
- [Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphology](https://mlsanity.com/paper/1906.04571/) (2019)
- [Crowdsourcing a Word-Emotion Association Lexicon](https://mlsanity.com/paper/1308.6297/) (2013)
- [Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias](https://mlsanity.com/paper/2004.12265/) (2020)
- [Adversarial Removal of Demographic Attributes from Text Data](https://mlsanity.com/paper/1808.06640/) (2018)
- [Learning Gender-Neutral Word Embeddings](https://mlsanity.com/paper/1809.01496/) (2018)

## Built on by (most cited first, 30 of 89)

- [Having Beer after Prayer? Measuring Cultural Bias in Large Language Models](https://mlsanity.com/paper/2305.14456/) (2023)
- [Probing Pre-Trained Language Models for Cross-Cultural Differences in Values](https://mlsanity.com/paper/2203.13722/) (2022)
- [Agree to Disagree: Diversity through Disagreement for Better Transferability](https://mlsanity.com/paper/2202.04414/) (2022)
- [How Ready are Pre-trained Abstractive Models and LLMs for Legal Case Judgement Summarization?](https://mlsanity.com/paper/2306.01248/) (2023)
- [Bootstrapping LLM-based Task-Oriented Dialogue Agents via Self-Talk](https://mlsanity.com/paper/2401.05033/) (2024)
- [Social Bias Probing: Fairness Benchmarking for Language Models](https://mlsanity.com/paper/2311.09090/) (2023)
- [The ROOTS Search Tool: Data Transparency for LLMs](https://mlsanity.com/paper/2302.14035/) (2023)
- [Choose Your Lenses: Flaws in Gender Bias Evaluation](https://mlsanity.com/paper/2210.11471/) (2022)
- [On the Independence of Association Bias and Empirical Fairness in Language Models](https://mlsanity.com/paper/2304.10153/) (2023)
- [Causally Testing Gender Bias in LLMs: A Case Study on Occupational Bias](https://mlsanity.com/paper/2212.10678/) (2022)
- [Sociodemographic Bias in Language Models: A Survey and Forward Path](https://mlsanity.com/paper/2306.08158/) (2023)
- [Debiasing Algorithm through Model Adaptation](https://mlsanity.com/paper/2310.18913/) (2023)
- [Looking for a Handsome Carpenter! Debiasing GPT-3 Job Advertisements](https://mlsanity.com/paper/2205.11374/) (2022)
- [Gender Biases and Where to Find Them: Exploring Gender Bias in Pre-Trained Transformer-based Language Models Using Movement Pruning](https://mlsanity.com/paper/2207.02463/) (2022)
- [Akal Badi ya Bias: An Exploratory Study of Gender Bias in Hindi Language Technology](https://mlsanity.com/paper/2405.06346/) (2024)
- [How Conservative are Language Models? Adapting to the Introduction of Gender-Neutral Pronouns](https://mlsanity.com/paper/2204.10281/) (2022)
- [Don't Forget About Pronouns: Removing Gender Bias in Language Models Without Losing Factual Gender Information](https://mlsanity.com/paper/2206.10744/) (2022)
- [Fairness in Language Models Beyond English: Gaps and Challenges](https://mlsanity.com/paper/2302.12578/) (2023)
- [Tokenization Matters: Navigating Data-Scarce Tokenization for Gender Inclusive Language Technologies](https://mlsanity.com/paper/2312.11779/) (2023)
- [''Fifty Shades of Bias'': Normative Ratings of Gender Bias in GPT Generated English Text](https://mlsanity.com/paper/2310.17428/) (2023)
- [From 'Showgirls' to 'Performers': Fine-tuning with Gender-inclusive Language for Bias Reduction in LLMs](https://mlsanity.com/paper/2407.04434/) (2024)
- [What the Harm? Quantifying the Tangible Impact of Gender Bias in Machine Translation with a Human-centered Study](https://mlsanity.com/paper/2410.00545/) (2024)
- [Investigating Bias in Multilingual Language Models: Cross-Lingual Transfer of Debiasing Techniques](https://mlsanity.com/paper/2310.10310/) (2023)
- [How Deep Is Representational Bias in LLMs? The Cases of Caste and Religion](https://mlsanity.com/paper/2508.03712/) (2025)
- [Social Bias in Large Language Models For Bangla: An Empirical Study on Gender and Religious Bias](https://mlsanity.com/paper/2407.03536/) (2024)
- [Metaethical Perspectives on 'Benchmarking' AI Ethics](https://mlsanity.com/paper/2204.05151/) (2022)
- [Women Are Beautiful, Men Are Leaders: Gender Stereotypes in Machine Translation and Language Modeling](https://mlsanity.com/paper/2311.18711/) (2023)
- [Measuring Gender Bias in West Slavic Language Models](https://mlsanity.com/paper/2304.05783/) (2023)
- [No Free Lunch: Retrieval-Augmented Generation Undermines Fairness in LLMs, Even for Vigilant Users](https://mlsanity.com/paper/2410.07589/) (2024)
- [In-Depth Look at Word Filling Societal Bias Measures](https://mlsanity.com/paper/2302.12640/) (2023)

## Abstract (arXiv)

Language can be used as a means of reproducing and enforcing harmful stereotypes and biases and has been analysed as such in numerous research. In this paper, we present a survey of 304 papers on gender bias in natural language processing. We analyse definitions of gender and its categories within social sciences and connect them to formal definitions of gender bias in NLP research. We survey lexica and datasets applied in research on gender bias and then compare and contrast approaches to detecting and mitigating gender bias. We find that research on gender bias suffers from four core limitations. 1) Most research treats gender as a binary variable neglecting its fluidity and continuity. 2) Most of the work has been conducted in monolingual setups for English or other high-resource languages. 3) Despite a myriad of papers on gender bias in NLP methods, we find that most of the newly developed algorithms do not test their models for bias and disregard possible ethical considerations of their work. 4) Finally, methodologies developed in this line of research are fundamentally flawed covering very limited definitions of gender bias and lacking evaluation baselines and pipelines. We suggest recommendations towards overcoming these limitations as a guide for future research.