# Word Embeddings via Causal Inference: Gender Bias Reducing and Semantic Information Preserving

arXiv: https://arxiv.org/abs/2112.05194 · HTML: https://mlsanity.com/paper/2112.05194/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 38 references that are in the map.
- First posted on arXiv 9 Dec 2021.

## Builds on (most cited first, 19 of 19)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them](https://mlsanity.com/paper/1903.03862/) (2019)
- [Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes](https://mlsanity.com/paper/1711.08412/) (2017)
- [Gender Bias in Contextualized Word Embeddings](https://mlsanity.com/paper/1904.03310/) (2019)
- [Avoiding Discrimination through Causal Reasoning](https://mlsanity.com/paper/1706.02744/) (2017)
- [SimLex-999: Evaluating Semantic Models with (Genuine) Similarity Estimation](https://mlsanity.com/paper/1408.3456/) (2014)
- [Learning Gender-Neutral Word Embeddings](https://mlsanity.com/paper/1809.01496/) (2018)
- [Black is to Criminal as Caucasian is to Police: Detecting and Removing Multiclass Bias in Word Embeddings](https://mlsanity.com/paper/1904.04047/) (2019)
- [Attenuating Bias in Word Vectors](https://mlsanity.com/paper/1901.07656/) (2019)
- [Gender-preserving Debiasing for Pre-trained Word Embeddings](https://mlsanity.com/paper/1906.00742/) (2019)
- [SimVerb-3500: A Large-Scale Evaluation Set of Verb Similarity](https://mlsanity.com/paper/1608.00869/) (2016)
- [Double-Hard Debias: Tailoring Word Embeddings for Gender Bias Mitigation](https://mlsanity.com/paper/2005.00965/) (2020)
- [A Causal Inference Method for Reducing Gender Bias in Word Embedding Relations](https://mlsanity.com/paper/1911.10787/) (2019)
- [Neutralizing Gender Bias in Word Embedding with Latent Disentanglement and Counterfactual Generation](https://mlsanity.com/paper/2004.03133/) (2020)
- [Debiasing Multilingual Word Embeddings: A Case Study of Three Indian Languages](https://mlsanity.com/paper/2107.10181/) (2021)
- [Partial Functional Linear Quantile Regression for Neuroimaging Data Analysis](https://mlsanity.com/paper/1511.00632/) (2015)
- [Sparse Wavelet Estimation in Quantile Regression with Multiple Functional Predictors](https://mlsanity.com/paper/1706.02353/) (2017)

## Built on by (most cited first, 11 of 11)

- [Fair Text-to-Image Diffusion via Fair Mapping](https://mlsanity.com/paper/2311.17695/) (2023)
- [Large Language Models and Causal Inference in Collaboration: A Survey](https://mlsanity.com/paper/2403.09606/) (2024)
- [NBIAS: A Natural Language Processing Framework for Bias Identification in Text](https://mlsanity.com/paper/2308.01681/) (2023)
- [Conformalized Fairness via Quantile Regression](https://mlsanity.com/paper/2210.02015/) (2022)
- [A Causal Explainable Guardrails for Large Language Models](https://mlsanity.com/paper/2405.04160/) (2024)
- [Diverse Target and Contribution Scheduling for Domain Generalization](https://mlsanity.com/paper/2309.16460/) (2023)
- [Unlocking Bias Detection: Leveraging Transformer-Based Models for Content Analysis](https://mlsanity.com/paper/2310.00347/) (2023)
- [REFINE-LM: Mitigating Language Model Stereotypes via Reinforcement Learning](https://mlsanity.com/paper/2408.09489/) (2024)
- [A Note on Bias to Complete](https://mlsanity.com/paper/2402.11710/) (2024)
- [Elucidate Gender Fairness in Singing Voice Transcription](https://mlsanity.com/paper/2308.02898/) (2023)
- [MeCaMIL: Causality-Aware Multiple Instance Learning for Fair and Interpretable Whole Slide Image Diagnosis](https://mlsanity.com/paper/2511.11004/) (2025)

## Abstract (arXiv)

With widening deployments of natural language processing (NLP) in daily life, inherited social biases from NLP models have become more severe and problematic. Previous studies have shown that word embeddings trained on human-generated corpora have strong gender biases that can produce discriminative results in downstream tasks. Previous debiasing methods focus mainly on modeling bias and only implicitly consider semantic information while completely overlooking the complex underlying causal structure among bias and semantic components. To address these issues, we propose a novel methodology that leverages a causal inference framework to effectively remove gender bias. The proposed method allows us to construct and analyze the complex causal mechanisms facilitating gender information flow while retaining oracle semantic information within word embeddings. Our comprehensive experiments show that the proposed method achieves state-of-the-art results in gender-debiasing tasks. In addition, our methods yield better performance in word similarity evaluation and various extrinsic downstream NLP tasks.