# Neutralizing Gender Bias in Word Embedding with Latent Disentanglement and Counterfactual Generation

arXiv: https://arxiv.org/abs/2004.03133 · HTML: https://mlsanity.com/paper/2004.03133/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Apr 2020.

## Built on by (most cited first, 10 of 10)

- [Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metrics](https://mlsanity.com/paper/2106.14574/) (2021)
- [Theories of "Gender" in NLP Bias Research](https://mlsanity.com/paper/2205.02526/) (2022)
- [Counterfactual Interventions Reveal the Causal Effect of Relative Clause Representations on Agreement Prediction](https://mlsanity.com/paper/2105.06965/) (2021)
- [Choose Your Lenses: Flaws in Gender Bias Evaluation](https://mlsanity.com/paper/2210.11471/) (2022)
- [Style Pooling: Automatic Text Style Obfuscation for Improved Classification Fairness](https://mlsanity.com/paper/2109.04624/) (2021)
- [Word Embeddings via Causal Inference: Gender Bias Reducing and Semantic Information Preserving](https://mlsanity.com/paper/2112.05194/) (2021)
- [Measuring Machine Learning Harms from Stereotypes Requires Understanding Who Is Harmed by Which Errors in What Ways](https://mlsanity.com/paper/2402.04420/) (2024)
- [Does Debiasing Inevitably Degrade the Model Performance](https://mlsanity.com/paper/2211.07350/) (2022)
- [Detecting Bias in Large Language Models: Fine-tuned KcBERT](https://mlsanity.com/paper/2403.10774/) (2024)
- [A Note on Bias to Complete](https://mlsanity.com/paper/2402.11710/) (2024)

## Abstract (arXiv)

Recent research demonstrates that word embeddings, trained on the human-generated corpus, have strong gender biases in embedding spaces, and these biases can result in the discriminative results from the various downstream tasks. Whereas the previous methods project word embeddings into a linear subspace for debiasing, we introduce a textitLatent Disentanglement method with a siamese auto-encoder structure with an adapted gradient reversal layer. Our structure enables the separation of the semantic latent information and gender latent information of given word into the disjoint latent dimensions. Afterwards, we introduce a textitCounterfactual Generation to convert the gender information of words, so the original and the modified embeddings can produce a gender-neutralized word embedding after geometric alignment regularization, without loss of semantic information. From the various quantitative and qualitative debiasing experiments, our method shows to be better than existing debiasing methods in debiasing word embeddings. In addition, Our method shows the ability to preserve semantic information during debiasing by minimizing the semantic information losses for extrinsic NLP downstream tasks.