mlsanityData to 7 Oct 2026

Gender-preserving Debiasing for Pre-trained Word Embeddings

arXiv 1906.00742 ↗First posted 3 Jun 2019cs.CL cs.LGcited by 61

Not on the map. Why?

Abstract (arXiv)

Word embeddings learnt from massive text collections have demonstrated significant levels of discriminative biases such as gender, racial or ethnic biases, which in turn bias the down-stream NLP applications that use those word embeddings. Taking gender-bias as a working example, we propose a debiasing method that preserves non-discriminative gender-related information, while removing stereotypical discriminative gender biases from pre-trained word embeddings. Specifically, we consider four types of information: feminine, masculine, gender-neutral and stereotypical, which represent the relationship between gender vs. bias, and propose a debiasing method that (a) preserves the gender-related information in feminine and masculine words, (b) preserves the neutrality in gender-neutral words, and (c) removes the biases from stereotypical words. Experimental results on several previously proposed benchmark datasets show that our proposed method can debias pre-trained word embeddings better than existing SoTA methods proposed for debiasing word embeddings while preserving gender-related but non-discriminative information.

Key facts. Cited by 61 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 15; 2022: 20; 2023: 12; 2024: 10; 2025: 3; 2026: 1.

61 papers.

Sort:

By area:

  1. 2021Debiasing Pre-trained Contextualised Embeddingscited by 107
  2. 2021An Interpretability Illusion for BERTcited by 94
  3. 2021A Survey on Gender Bias in Natural Language Processingcited by 89
  4. 2021FairFil: Contrastive Neural Debiasing Method for Pretrained Text Encoderscited by 75
  5. 2022Theories of "Gender" in NLP Bias Researchcited by 62
  6. 2021Unmasking the Mask -- Evaluating Social Biases in Masked Language Modelscited by 60
  7. 2022Gender Bias in Masked Language Models for Multiple Languagescited by 60
  8. 2022Debiasing isn't enough! -- On the Effectiveness of Debiasing MLMs and their Social Biases in Downstream Taskscited by 48
  9. 2022Generalizing to the Future: Mitigating Entity Bias in Fake News Detectioncited by 38
  10. 2022Choose Your Lenses: Flaws in Gender Bias Evaluationcited by 29
  11. 2023Sociodemographic Bias in Language Models: A Survey and Forward Pathcited by 27
  12. 2023Survey of Social Bias in Vision-Language Modelscited by 26
38 more
  1. 2021Dictionary-based Debiasing of Pre-trained Word Embeddingscited by 24
  2. 2021Intersectional Bias in Causal Language Modelscited by 24
  3. 2022Debiasing Methods for Fairer Neural Models in Vision and Language Research: A Surveycited by 22
  4. 2022Evaluating the Diversity, Equity and Inclusion of NLP Technology: A Case Study for Indian Languagescited by 20
  5. 2022Gender Biases and Where to Find Them: Exploring Gender Bias in Pre-Trained Transformer-based Language Models Using Movement Pruningcited by 20
  6. 2022ADEPT: A DEbiasing PrompT Frameworkcited by 20
  7. 2021Obtaining Better Static Word Embeddings Using Contextual Embedding Modelscited by 18
  8. 2022Sense Embeddings are also Biased--Evaluating Social Biases in Static and Contextualised Sense Embeddingscited by 18
  9. 2022How to keep text private? A systematic review of deep learning methods for privacy-preserving natural language processingcited by 16
  10. 2022Don't Forget About Pronouns: Removing Gender Bias in Language Models Without Losing Factual Gender Informationcited by 16
  11. 2023Bias A-head? Analyzing Bias in Transformer-Based Language Model Attention Headscited by 13
  12. 2021Style Pooling: Automatic Text Style Obfuscation for Improved Classification Fairnesscited by 12
  13. 2023Gender-tuning: Empowering Fine-tuning for Debiasing Pre-trained Language Modelscited by 12
  14. 2023In-Contextual Gender Bias Suppression for Large Language Modelscited by 12
  15. 2024Bias Vector: Mitigating Biases in Language Models with Task Arithmetic Approachcited by 12
  16. 2021Word Embeddings via Causal Inference: Gender Bias Reducing and Semantic Information Preservingcited by 11
  17. 2022You Are What You Write: Preserving Privacy in the Era of Large Language Modelscited by 11
  18. 2022Combining Static and Contextualised Multilingual Embeddingscited by 8
  19. 2023Comparing Intrinsic Gender Bias Evaluation Measures without using Human Annotated Examplescited by 8
  20. 2022A Survey on Word Meta-Embedding Learningcited by 7
  21. 2024'Since Lawyers are Males..': Examining Implicit Gender Bias in Hindi Language Generation by LLMscited by 7
  22. 2025Eliminating Inductive Bias in Reward Models with Information-Theoretic Guidancecited by 7
  23. 2021Marked Attribute Bias in Natural Language Inferencecited by 6
  24. 2022Gender Bias in Meta-Embeddingscited by 6
  25. 2022Learning Meta Word Embeddings by Unsupervised Weighted Concatenation of Source Embeddingscited by 5
  26. 2022Toward Understanding Bias Correlations for Mitigation in NLPcited by 5
  27. 2022How people talk about each other: Modeling Generalized Intergroup Bias and Emotioncited by 5
  28. 2023Solving NLP Problems through Human-System Collaboration: A Discussion-based Approachcited by 5
  29. 2024The Gaps between Pre-train and Downstream Settings in Bias Evaluation and Debiasingcited by 5
  30. 2024Evolving to the Future: Unseen Event Adaptive Fake News Detection on Social Mediacited by 5
  31. 2021On the Universality of Deep Contextual Language Modelscited by 4
  32. 2023A Neighbourhood-Aware Differential Privacy Mechanism for Static Word Embeddingscited by 4
  33. 2023A Predictive Factor Analysis of Social Biases and Task-Performance in Pretrained Masked Language Modelscited by 4
  34. 2024Measuring Machine Learning Harms from Stereotypes Requires Understanding Who Is Harmed by Which Errors in What Wayscited by 4
  35. 2025SCDTour: Embedding Axis Ordering and Merging for Interpretable Semantic Change Detectioncited by 4
  36. 2024The Life Cycle of Large Language Models: A Review of Biases in Educationcited by 3
  37. 2021Identifying and Mitigating Gender Bias in Hyperbolic Word Embeddingscited by 2
  38. 2021Deep Learning for Bias Detection: From Inception to Deploymentcited by 2

50 most cited of 61 All 61 (JSON)