# VERB: Visualizing and Interpreting Bias Mitigation Techniques for Word Representations

arXiv: https://arxiv.org/abs/2104.02797 · HTML: https://mlsanity.com/paper/2104.02797/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 69 references that are in the map.
- First posted on arXiv 6 Apr 2021.

## Builds on (most cited first, 30 of 31)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection](https://mlsanity.com/paper/2004.07667/) (2020)
- [Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them](https://mlsanity.com/paper/1903.03862/) (2019)
- [Learning Gender-Neutral Word Embeddings](https://mlsanity.com/paper/1809.01496/) (2018)
- [On Measuring and Mitigating Biased Inferences of Word Embeddings](https://mlsanity.com/paper/1908.09369/) (2019)
- [FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learning](https://mlsanity.com/paper/1904.05419/) (2019)
- [Attenuating Bias in Word Vectors](https://mlsanity.com/paper/1901.07656/) (2019)
- [A Survey of Visual Analytics Techniques for Machine Learning](https://mlsanity.com/paper/2008.09632/) (2020)
- [Towards Better Analysis of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1604.07043/) (2016)
- [Embedding Projector: Interactive Visualization and Interpretation of Embeddings](https://mlsanity.com/paper/1611.05469/) (2016)
- [OSCaR: Orthogonal Subspace Correction and Rectification of Biases in Word Embeddings](https://mlsanity.com/paper/2007.00049/) (2020)
- [Visual Analytics for Explainable Deep Learning](https://mlsanity.com/paper/1804.02527/) (2018)
- [Towards Better Analysis of Machine Learning Models: A Visual Analytics Perspective](https://mlsanity.com/paper/1702.01226/) (2017)
- [Double-Hard Debias: Tailoring Word Embeddings for Gender Bias Mitigation](https://mlsanity.com/paper/2005.00965/) (2020)
- [OoDAnalyzer: Interactive Analysis of Out-of-Distribution Samples](https://mlsanity.com/paper/2002.03103/) (2020)
- [EmbeddingVis: A Visual Analytics Approach to Comparative Network Embedding Inspection](https://mlsanity.com/paper/1808.09074/) (2018)
- [ConfusionFlow: A model-agnostic visualization for temporal analysis of classifier confusion](https://mlsanity.com/paper/1910.00969/) (2019)
- [embComp: Visual Interactive Comparison of Vector Embeddings](https://mlsanity.com/paper/1911.01542/) (2019)
- [Diagnosing Concept Drift with Visual Analytics](https://mlsanity.com/paper/2007.14372/) (2020)
- [Online Visual Analytics of Text Streams](https://mlsanity.com/paper/1512.04042/) (2015)
- [DebIE: A Platform for Implicit and Explicit Debiasing of Word Embedding Spaces](https://mlsanity.com/paper/2103.06598/) (2021)
- [TopoAct: Visually Exploring the Shape of Activations in Deep Learning](https://mlsanity.com/paper/1912.06332/) (2019)

## Built on by (most cited first, 3 of 3)

- [Explainable Mapper: Charting LLM Embedding Spaces Using Perturbation-Based Explanation and Verification Agents](https://mlsanity.com/paper/2507.18607/) (2025)
- [A methodology to characterize bias and harmful stereotypes in natural language processing in Latin America](https://mlsanity.com/paper/2207.06591/) (2022)
- [AI Fairness: from Principles to Practice](https://mlsanity.com/paper/2207.09833/) (2022)

## Abstract (arXiv)

Word vector embeddings have been shown to contain and amplify biases in data they are extracted from. Consequently, many techniques have been proposed to identify, mitigate, and attenuate these biases in word representations. In this paper, we utilize interactive visualization to increase the interpretability and accessibility of a collection of state-of-the-art debiasing techniques. To aid this, we present Visualization of Embedding Representations for deBiasing system ("VERB"), an open-source web-based visualization tool that helps the users gain a technical understanding and visual intuition of the inner workings of debiasing techniques, with a focus on their geometric properties. In particular, VERB offers easy-to-follow use cases in exploring the effects of these debiasing techniques on the geometry of high-dimensional word vectors. To help understand how various debiasing techniques change the underlying geometry, VERB decomposes each technique into interpretable sequences of primitive transformations and highlights their effect on the word vectors using dimensionality reduction and interactive visual exploration. VERB is designed to target natural language processing (NLP) practitioners who are designing decision-making systems on top of word embeddings, and also researchers working with fairness and ethics of machine learning systems in NLP. It can also serve as a visual medium for education, which helps an NLP novice to understand and mitigate biases in word embeddings.