# Exploring the Linear Subspace Hypothesis in Gender Bias Mitigation

arXiv: https://arxiv.org/abs/2009.09435 · HTML: https://mlsanity.com/paper/2009.09435/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Sep 2020.

## Built on by (most cited first, 25 of 25)

- [Linear Adversarial Concept Erasure](https://mlsanity.com/paper/2201.12091/) (2022)
- [Kernelized Concept Erasure](https://mlsanity.com/paper/2201.12191/) (2022)
- [The Low-Dimensional Linear Geometry of Contextualized Word Representations](https://mlsanity.com/paper/2105.07109/) (2021)
- [Overcoming Sparsity Artifacts in Crosscoders to Interpret Chat-Tuning](https://mlsanity.com/paper/2504.02922/) (2025)
- [A Word on Machine Ethics: A Response to Jiang et al. (2021)](https://mlsanity.com/paper/2111.04158/) (2021)
- [Controllable Context Sensitivity and the Knob Behind It](https://mlsanity.com/paper/2411.07404/) (2024)
- [Debiasing Methods for Fairer Neural Models in Vision and Language Research: A Survey](https://mlsanity.com/paper/2211.05617/) (2022)
- [Better Hit the Nail on the Head than Beat around the Bush: Removing Protected Attributes with a Single Projection](https://mlsanity.com/paper/2212.04273/) (2022)
- [Towards a theory of model distillation](https://mlsanity.com/paper/2403.09053/) (2024)
- [Assessing the Reliability of Word Embedding Gender Bias Measures](https://mlsanity.com/paper/2109.04732/) (2021)
- [Gumbel Counterfactual Generation From Language Models](https://mlsanity.com/paper/2411.07180/) (2024)
- [Emergence of Linear Truth Encodings in Language Models](https://mlsanity.com/paper/2510.15804/) (2025)
- [How Reliable are Causal Probing Interventions?](https://mlsanity.com/paper/2408.15510/) (2024)
- [The Information Geometry of Softmax: Probing and Steering](https://mlsanity.com/paper/2602.15293/) (2026)
- [WordBias: An Interactive Visual Tool for Discovering Intersectional Biases Encoded in Word Embeddings](https://mlsanity.com/paper/2103.03598/) (2021)
- [Evolution of Concepts in Language Model Pre-Training](https://mlsanity.com/paper/2509.17196/) (2025)
- [Removing Spurious Concepts from Neural Network Representations via Joint Subspace Estimation](https://mlsanity.com/paper/2310.11991/) (2023)
- [On a Benefit of Mask Language Modeling: Robustness to Simplicity Bias](https://mlsanity.com/paper/2110.05301/) (2021)
- [Competence-Based Analysis of Language Models](https://mlsanity.com/paper/2303.00333/) (2023)
- [Nonlinear Concept Erasure: a Density Matching Approach](https://mlsanity.com/paper/2507.12341/) (2025)
- [This Reads Like That: Deep Learning for Interpretable Natural Language Processing](https://mlsanity.com/paper/2310.17010/) (2023)
- [Unsupervised Detection of Contextualized Embedding Bias with Application to Ideology](https://mlsanity.com/paper/2212.07547/) (2022)
- [Towards Fair and Explainable AI using a Human-Centered AI Approach](https://mlsanity.com/paper/2306.07427/) (2023)
- [Do Large Language Models know who did what to whom?](https://mlsanity.com/paper/2504.16884/) (2025)
- [MUtE: A Dual Framework for Concept Erasure and Counterfactual Interventions](https://mlsanity.com/paper/2609.11253/) (2026)

## Abstract (arXiv)

Bolukbasi et al. (2016) presents one of the first gender bias mitigation techniques for word representations. Their method takes pre-trained word representations as input and attempts to isolate a linear subspace that captures most of the gender bias in the representations. As judged by an analogical evaluation task, their method virtually eliminates gender bias in the representations. However, an implicit and untested assumption of their method is that the bias subspace is actually linear. In this work, we generalize their method to a kernelized, nonlinear version. We take inspiration from kernel principal component analysis and derive a nonlinear bias isolation technique. We discuss and overcome some of the practical drawbacks of our method for non-linear gender bias mitigation in word representations and analyze empirically whether the bias subspace is actually linear. Our analysis shows that gender bias is in fact well captured by a linear subspace, justifying the assumption of Bolukbasi et al. (2016).