# Investigating Gender Bias in BERT

arXiv: https://arxiv.org/abs/2009.05021 · HTML: https://mlsanity.com/paper/2009.05021/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Sep 2020.

## Built on by (most cited first, 30 of 40)

- [Bias Out-of-the-Box: An Empirical Analysis of Intersectional Occupational Biases in Popular Generative Language Models](https://mlsanity.com/paper/2102.04130/) (2021)
- [A Survey on Gender Bias in Natural Language Processing](https://mlsanity.com/paper/2112.14168/) (2021)
- [What's in the Box? A Preliminary Analysis of Undesirable Content in the Common Crawl Corpus](https://mlsanity.com/paper/2105.02732/) (2021)
- [Theories of "Gender" in NLP Bias Research](https://mlsanity.com/paper/2205.02526/) (2022)
- [One Teacher is Enough? Pre-trained Language Model Distillation from Multiple Teachers](https://mlsanity.com/paper/2106.01023/) (2021)
- [Benefits and Harms of Large Language Models in Digital Mental Health](https://mlsanity.com/paper/2311.14693/) (2023)
- [Quantifying Societal Bias Amplification in Image Captioning](https://mlsanity.com/paper/2203.15395/) (2022)
- [Parameter-efficient Modularised Bias Mitigation via AdapterFusion](https://mlsanity.com/paper/2302.06321/) (2023)
- [Measuring and Improving BERT's Mathematical Abilities by Predicting the Order of Reasoning](https://mlsanity.com/paper/2106.03921/) (2021)
- [Gender Bias in BERT -- Measuring and Analysing Biases through Sentiment Rating in a Realistic Downstream Classification Task](https://mlsanity.com/paper/2306.15298/) (2023)
- [The Media Bias Taxonomy: A Systematic Literature Review on the Forms and Automated Detection of Media Bias](https://mlsanity.com/paper/2312.16148/) (2023)
- [Towards an Enhanced Understanding of Bias in Pre-trained Neural Language Models: A Survey with Special Emphasis on Affective Bias](https://mlsanity.com/paper/2204.10365/) (2022)
- [Finding Structural Knowledge in Multimodal-BERT](https://mlsanity.com/paper/2203.09306/) (2022)
- [Measuring Harmful Representations in Scandinavian Language Models](https://mlsanity.com/paper/2211.11678/) (2022)
- [Toward Understanding Bias Correlations for Mitigation in NLP](https://mlsanity.com/paper/2205.12391/) (2022)
- [Power of Explanations: Towards automatic debiasing in hate speech detection](https://mlsanity.com/paper/2209.09975/) (2022)
- [Bridging the Fairness Gap: Enhancing Pre-trained Models with LLM-Generated Sentences](https://mlsanity.com/paper/2501.06795/) (2025)
- [On the Universality of Deep Contextual Language Models](https://mlsanity.com/paper/2109.07140/) (2021)
- [Unintended Bias in Language Model-driven Conversational Recommendation](https://mlsanity.com/paper/2201.06224/) (2022)
- [Gender Bias in Text: Labeled Datasets and Lexicons](https://mlsanity.com/paper/2201.08675/) (2022)
- [Artefact Retrieval: Overview of NLP Models with Knowledge Base Access](https://mlsanity.com/paper/2201.09651/) (2022)
- [Identifying and Measuring Token-Level Sentiment Bias in Pre-trained Language Models with Prompts](https://mlsanity.com/paper/2204.07289/) (2022)
- [LoNLI: An Extensible Framework for Testing Diverse Logical Reasoning Capabilities for NLI](https://mlsanity.com/paper/2112.02333/) (2021)
- [Measuring CLEVRness: Blackbox testing of Visual Reasoning Models](https://mlsanity.com/paper/2202.12162/) (2022)
- [Are fairness metric scores enough to assess discrimination biases in machine learning?](https://mlsanity.com/paper/2306.05307/) (2023)
- [An investigation of structures responsible for gender bias in BERT and DistilBERT](https://mlsanity.com/paper/2401.06495/) (2024)
- [NLP4PBM: A Systematic Review on Process Extraction using Natural Language Processing with Rule-based, Machine and Deep Learning Methods](https://mlsanity.com/paper/2409.13738/) (2024)
- [Debiasing Gender Bias in Information Retrieval Models](https://mlsanity.com/paper/2208.01755/) (2022)
- [Does Debiasing Inevitably Degrade the Model Performance](https://mlsanity.com/paper/2211.07350/) (2022)
- [Gender Bias in Transformer Models: A comprehensive survey](https://mlsanity.com/paper/2306.10530/) (2023)

## Abstract (arXiv)

Contextual language models (CLMs) have pushed the NLP benchmarks to a new height. It has become a new norm to utilize CLM provided word embeddings in downstream tasks such as text classification. However, unless addressed, CLMs are prone to learn intrinsic gender-bias in the dataset. As a result, predictions of downstream NLP models can vary noticeably by varying gender words, such as replacing "he" to "she", or even gender-neutral words. In this paper, we focus our analysis on a popular CLM, i.e., BERT. We analyse the gender-bias it induces in five downstream tasks related to emotion and sentiment intensity prediction. For each task, we train a simple regressor utilizing BERT's word embeddings. We then evaluate the gender-bias in regressors using an equity evaluation corpus. Ideally and from the specific design, the models should discard gender informative features from the input. However, the results show a significant dependence of the system's predictions on gender-particular words and phrases. We claim that such biases can be reduced by removing genderspecific features from word embedding. Hence, for each layer in BERT, we identify directions that primarily encode gender information. The space formed by such directions is referred to as the gender subspace in the semantic space of word embeddings. We propose an algorithm that finds fine-grained gender directions, i.e., one primary direction for each BERT layer. This obviates the need of realizing gender subspace in multiple dimensions and prevents other crucial information from being omitted. Experiments show that removing embedding components in such directions achieves great success in reducing BERT-induced bias in the downstream tasks.