mlsanityData to 7 Oct 2026

Unmasking Contextual Stereotypes: Measuring and Mitigating BERT's Gender Bias

arXiv 2010.14534 ↗First posted 27 Oct 2020cs.CLcited by 82

Not on the map. Why?

Abstract (arXiv)

Contextualized word embeddings have been replacing standard embeddings as the representational knowledge source of choice in NLP systems. Since a variety of biases have previously been found in standard word embeddings, it is crucial to assess biases encoded in their replacements as well. Focusing on BERT (Devlin et al., 2018), we measure gender bias by studying associations between gender-denoting target words and names of professions in English and German, comparing the findings with real-world workforce statistics. We mitigate bias by fine-tuning BERT on the GAP corpus (Webster et al., 2018), after applying Counterfactual Data Substitution (CDS) (Maudslay et al., 2019). We show that our method of measuring bias is appropriate for languages such as English, but not for languages with a rich morphology and gender-marking, such as German. Our results highlight the importance of investigating bias and mitigation techniques cross-linguistically, especially in view of the current emphasis on large-scale, multilingual language models.

Key facts. Cited by 82 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 16; 2023: 21; 2024: 16; 2025: 19; 2026: 4.

82 papers.

Sort:

By area:

  1. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  2. 2021Sustainable Modular Debiasing of Language Modelscited by 98
  3. 2023Language Model Behavior: A Comprehensive Surveycited by 92
  4. 2024Large Language Models are Geographically Biasedcited by 91
  5. 2021A Survey on Gender Bias in Natural Language Processingcited by 89
  6. 2022Theories of "Gender" in NLP Bias Researchcited by 62
  7. 2022Gender Bias in Masked Language Models for Multiple Languagescited by 60
  8. 2024Fairness in Large Language Models: A Taxonomic Surveycited by 57
  9. 2022Entropy-based Attention Regularization Frees Unintended Bias Mitigation from Listscited by 54
  10. 2022Debiasing isn't enough! -- On the Effectiveness of Debiasing MLMs and their Social Biases in Downstream Taskscited by 48
  11. 2023Benefits and Harms of Large Language Models in Digital Mental Healthcited by 39
  12. 2022MABEL: Attenuating Gender Bias using Textual Entailment Datacited by 31
38 more
  1. 2022Choose Your Lenses: Flaws in Gender Bias Evaluationcited by 29
  2. 2023On the Independence of Association Bias and Empirical Fairness in Language Modelscited by 29
  3. 2023Sociodemographic Bias in Language Models: A Survey and Forward Pathcited by 27
  4. 2021Understanding and Countering Stereotypes: A Computational Approach to the Stereotype Content Modelcited by 23
  5. 2023Gender Bias in BERT -- Measuring and Analysing Biases through Sentiment Rating in a Realistic Downstream Classification Taskcited by 21
  6. 2021Measuring Fairness with Biased Rulers: A Survey on Quantifying Biases in Pretrained Language Modelscited by 20
  7. 2025Fairness through Difference Awareness: Measuring Desired Group Discrimination in LLMscited by 20
  8. 2022Naturalistic Causal Probing for Morpho-Syntaxcited by 19
  9. 2024Large Language Model Bias Mitigation from the Perspective of Knowledge Editingcited by 16
  10. 2024Multilingual Text-to-Image Generation Magnifies Gender Stereotypes and Prompt Engineering May Not Help Youcited by 15
  11. 2023CALM : A Multi-task Benchmark for Comprehensive Assessment of Language Model Biascited by 13
  12. 2024Identifying and Mitigating Social Bias Knowledge in Language Modelscited by 13
  13. 2022Detecting Unintended Social Bias in Toxic Language Datasetscited by 12
  14. 2023Multilingual large language models leak human stereotypes across language boundariescited by 12
  15. 2023Tackling Bias in Pre-trained Language Models: Current Trends and Under-represented Societiescited by 11
  16. 2022Analyzing Gender Representation in Multilingual Modelscited by 10
  17. 2023Measuring Gender Bias in West Slavic Language Modelscited by 10
  18. 2024Potential and Challenges of Model Editing for Social Debiasingcited by 10
  19. 2025A Comprehensive Analysis of Large Language Model Outputs: Similarity, Diversity, and Biascited by 10
  20. 2023SensePOLAR: Word sense aware interpretability for pre-trained contextual word embeddingscited by 9
  21. 2022Towards an Enhanced Understanding of Bias in Pre-trained Neural Language Models: A Survey with Special Emphasis on Affective Biascited by 8
  22. 2022FairDistillation: Mitigating Stereotyping in Language Modelscited by 8
  23. 2022HERB: Measuring Hierarchical Regional Bias in Pre-trained Language Modelscited by 8
  24. 2023BiasTestGPT: Using ChatGPT for Social Bias Testing of Language Modelscited by 8
  25. 2023Logic Against Bias: Textual Entailment Mitigates Stereotypical Sentence Reasoningcited by 7
  26. 2021Social Norm Bias: Residual Harms of Fairness-Aware Algorithmscited by 6
  27. 2025Rethinking LLM Bias Probing Using Lessons from the Social Sciencescited by 6
  28. 2023FairPy: A Toolkit for Evaluation of Prediction Biases and their Mitigation in Large Language Modelscited by 5
  29. 2024Leveraging Large Language Models to Measure Gender Representation Bias in Gendered Language Corporacited by 5
  30. 2024Understanding the Interplay of Scale, Data, and Bias in Language Models: A Case Study with BERTcited by 5
  31. 2025LangFair: A Python Package for Assessing Bias and Fairness in Large Language Model Use Casescited by 5
  32. 2025Assumed Identities: Quantifying Gender Bias in Machine Translation of Gender-Ambiguous Occupational Termscited by 5
  33. 2023Trade-Offs Between Fairness and Privacy in Language Modelingcited by 4
  34. 2023How Different Is Stereotypical Bias Across Languages?cited by 4
  35. 2023Unlocking Bias Detection: Leveraging Transformer-Based Models for Content Analysiscited by 4
  36. 2024Exploring Bengali Religious Dialect Biases in Large Language Models with Evaluation Perspectivescited by 4
  37. 2025LLM Bias Detection and Mitigation through the Lens of Desired Distributionscited by 4
  38. 2022Efficient Gender Debiasing of Pre-trained Indic Language Modelscited by 3

50 most cited of 82 All 82 (JSON)