mlsanityData to 7 Oct 2026

Don't Forget About Pronouns: Removing Gender Bias in Language Models Without Losing Factual Gender Information

arXiv 2206.10744 ↗First posted 21 Jun 2022cs.CL cs.AIcited by 16

Not on the map. Why?

Abstract (arXiv)

The representations in large language models contain multiple types of gender information. We focus on two types of such signals in English texts: factual gender information, which is a grammatical or semantic property, and gender bias, which is the correlation between a word and specific gender. We can disentangle the model's embeddings and identify components encoding both types of information with probing. We aim to diminish the stereotypical bias in the representations while preserving the factual gender signal. Our filtering method shows that it is possible to decrease the bias of gender-neutral profession names without significant deterioration of language modeling capabilities. The findings can be applied to language generation to mitigate reliance on stereotypes while preserving gender agreement in coreferences.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 18 of its 32 references that are in the map. First posted on arXiv 21 Jun 2022.

Builds on

18 of 32 references.

References · 18

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  4. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  5. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  6. 2016The Mythos of Model Interpretabilitycited by 716
  7. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
  8. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 657
  9. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 442
  10. 2019Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Settingcited by 346
  11. 2019Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Themcited by 268
  12. 2019Evaluating Gender Bias in Machine Translationcited by 247
6 more
  1. 2018Learning Gender-Neutral Word Embeddingscited by 169
  2. 2021A Survey on Gender Bias in Natural Language Processingcited by 89
  3. 2019Gender-preserving Debiasing for Pre-trained Word Embeddingscited by 61
  4. 2019Reducing Gender Bias in Word-Level Language Models with a Gender-Equalizing Loss Functioncited by 57
  5. 2021Stereotype and Skew: Quantifying Gender Bias in Pre-trained and Fine-tuned Language Modelscited by 57
  6. 2020Introducing Orthogonal Constraint in Structural Probescited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 5; 2024: 4; 2025: 5; 2026: 1.

16 papers.

Sort:

By area:

  1. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  2. 2023A Survey on Fairness in Large Language Modelscited by 91
  3. 2024Digital Forgetting in Large Language Models: A Survey of Unlearning Methodscited by 38
  4. 2024Model Editing as a Robust and Denoised variant of DPO: A Case Study on Toxicitycited by 33
  5. 2024GenderBias-VL: Benchmarking Gender Bias in Vision Language Models via Counterfactual Probingcited by 31
  6. 2023Sociodemographic Bias in Language Models: A Survey and Forward Pathcited by 27
  7. 2023Debiasing Algorithm through Model Adaptationcited by 24
  8. 2025BiasEdit: Debiasing Stereotyped Language Models via Model Editingcited by 18
  9. 2023Tackling Bias in Pre-trained Language Models: Current Trends and Under-represented Societiescited by 11
  10. 2025Assumed Identities: Quantifying Gender Bias in Machine Translation of Gender-Ambiguous Occupational Termscited by 5
  11. 2022Undesirable Biases in NLP: Addressing Challenges of Measurementcited by 4
  12. 2025LLM Bias Detection and Mitigation through the Lens of Desired Distributionscited by 4
4 more
  1. 2024The Life Cycle of Large Language Models: A Review of Biases in Educationcited by 3
  2. 2025Dual Debiasing: Remove Stereotypes and Keep Factual Gender for Fair Language Modeling and Translationcited by 2
  3. 2025Robust Bias Detection in MLMs and its Application to Human Trait Ratingscited by 1
  4. 2026GKnow: Measuring the Entanglement of Gender Bias and Factual Gendercited by 0