mlsanityData to 14 Oct 2026

Double-Hard Debias: Tailoring Word Embeddings for Gender Bias Mitigation

arXiv 2005.00965 ↗First posted 3 May 2020cs.CL cs.LGcited by 36

Not on the map. Why?

Abstract (arXiv)

Word embeddings derived from human-generated corpora inherit strong gender bias which can be further amplified by downstream models. Some commonly adopted debiasing approaches, including the seminal Hard Debias algorithm, apply post-processing procedures that project pre-trained word embeddings into a subspace orthogonal to an inferred gender subspace. We discover that semantic-agnostic corpus regularities such as word frequency captured by the word embeddings negatively impact the performance of these algorithms. We propose a simple but effective technique, Double Hard Debias, which purifies the word embeddings against such corpus regularities prior to inferring and removing the gender subspace. Experiments on three bias mitigation benchmarks show that our approach preserves the distributional semantics of the pre-trained word embeddings while reducing gender bias to a significantly larger degree than prior approaches.

Key facts. Cited by 36 papers in the map (data through 14 Oct 2026). First posted on arXiv 3 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 10; 2022: 8; 2023: 6; 2024: 4; 2025: 5; 2026: 3.

36 papers.

Sort:

By area:

  1. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 435
  2. 2021Mitigating Language-Dependent Ethnic Bias in BERTcited by 87
  3. 2022Theories of "Gender" in NLP Bias Researchcited by 62
  4. 2021Low Frequency Names Exhibit Bias and Overfitting in Contextualizing Language Modelscited by 39
  5. 2025Steering Out-of-Distribution Generalization with Concept Ablation Fine-Tuningcited by 39
  6. 2023Shielded Representations: Protecting Sensitive Attributes Through Iterative Gradient-Based Projectioncited by 25
  7. 2022Gender Bias in Word Embeddings: A Comprehensive Analysis of Frequency, Syntax, and Semanticscited by 24
  8. 2022Debiasing Methods for Fairer Neural Models in Vision and Language Research: A Surveycited by 22
  9. 2021Does Robustness Improve Fairness? Approaching Fairness with Word Substitution Robustness Methods for Text Classificationcited by 20
  10. 2022Sense Embeddings are also Biased--Evaluating Social Biases in Static and Contextualised Sense Embeddingscited by 18
  11. 2022The Birth of Bias: A case study on the evolution of gender bias in an English language modelcited by 16
  12. 2023Learning to Generate Equitable Text in Dialogue from Biased Training Datacited by 12
24 more
  1. 2024Bias Vector: Mitigating Biases in Language Models with Task Arithmetic Approachcited by 12
  2. 2021Word Embeddings via Causal Inference: Gender Bias Reducing and Semantic Information Preservingcited by 11
  3. 2023Large Human Language Models: A Need and the Challengescited by 11
  4. 2026Position: LLMs Must Use Functor-Based and RAG-Driven Bias Mitigation for Fairnesscited by 10
  5. 2022Fair NLP Models with Differentially Private Text Encoderscited by 9
  6. 2026CatRAG: Functor-Guided Structural Debiasing with Retrieval Augmentation for Fair LLMscited by 9
  7. 2021WordBias: An Interactive Visual Tool for Discovering Intersectional Biases Encoded in Word Embeddingscited by 8
  8. 2021Marked Attribute Bias in Natural Language Inferencecited by 6
  9. 2021Estimating and Maximizing Mutual Information for Knowledge Distillationcited by 6
  10. 2022Gender Bias in Meta-Embeddingscited by 6
  11. 2025Attention Pruning: Automated Fairness Repair of Language Models via Surrogate Simulated Annealingcited by 5
  12. 2023Fair Text Classification with Wasserstein Independencecited by 4
  13. 2025On the Mutual Influence of Gender and Occupation in LLM Representationscited by 4
  14. 2025Chain-of-Thought Driven Adversarial Scenario Extrapolation for Robust Language Modelscited by 4
  15. 2021VERB: Visualizing and Interpreting Bias Mitigation Techniques for Word Representationscited by 3
  16. 2024Algorithmic Fairness: A Tolerance Perspectivecited by 2
  17. 2021Preliminary Steps Towards Federated Sentiment Classificationcited by 1
  18. 2021Adversarial Examples Generation for Reducing Implicit Gender Bias in Pre-trained Modelscited by 1
  19. 2022A Multibias-mitigated and Sentiment Knowledge Enriched Transformer for Debiasing in Multimodal Conversational Emotion Recognitioncited by 1
  20. 2023Towards Fair and Explainable AI using a Human-Centered AI Approachcited by 1
  21. 2024Mitigating Gender Bias in Contextual Word Embeddingscited by 1
  22. 2025Exploring and Mitigating Gender Bias in Encoder-Based Transformer Modelscited by 1
  23. 2023Detecting and Mitigating Indirect Stereotypes in Word Embeddingscited by 0
  24. 2026A Representation-Level Assessment of Bias Mitigation in Foundation Modelscited by 0