# A Source-Criticism Debiasing Method for GloVe Embeddings

arXiv: https://arxiv.org/abs/2106.13382 · HTML: https://mlsanity.com/paper/2106.13382/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 14 references that are in the map.
- First posted on arXiv 25 Jun 2021.

## Builds on (most cited first, 12 of 12)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them](https://mlsanity.com/paper/1903.03862/) (2019)
- [Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes](https://mlsanity.com/paper/1711.08412/) (2017)
- [Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems](https://mlsanity.com/paper/1805.04508/) (2018)
- [Explainable Machine Learning in Deployment](https://mlsanity.com/paper/1909.06342/) (2019)
- [Learning Gender-Neutral Word Embeddings](https://mlsanity.com/paper/1809.01496/) (2018)
- [FastIF: Scalable Influence Functions for Efficient Model Interpretation and Debugging](https://mlsanity.com/paper/2012.15781/) (2020)
- [Understanding the Origins of Bias in Word Embeddings](https://mlsanity.com/paper/1810.03611/) (2018)

## Built on by (most cited first, 1 of 1)

- [Debiasing Methods for Fairer Neural Models in Vision and Language Research: A Survey](https://mlsanity.com/paper/2211.05617/) (2022)

## Abstract (arXiv)

It is well-documented that word embeddings trained on large public corpora consistently exhibit known human social biases. Although many methods for debiasing exist, almost all fixate on completely eliminating biased information from the embeddings and often diminish training set size in the process. In this paper, we present a simple yet effective method for debiasing GloVe word embeddings (Pennington et al., 2014) which works by incorporating explicit information about training set bias rather than removing biased data outright. Our method runs quickly and efficiently with the help of a fast bias gradient approximation method from Brunet et al. (2019). As our approach is akin to the notion of 'source criticism' in the humanities, we term our method Source-Critical GloVe (SC-GloVe). We show that SC-GloVe reduces the effect size on Word Embedding Association Test (WEAT) sets without sacrificing training data or TOP-1 performance.