# What Happens to a Dataset Transformed by a Projection-based Concept Removal Method?

arXiv: https://arxiv.org/abs/2403.16142 · HTML: https://mlsanity.com/paper/2403.16142/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 25 references that are in the map.
- First posted on arXiv 24 Mar 2024.

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection](https://mlsanity.com/paper/2004.07667/) (2020)
- [LEACE: Perfect linear concept erasure in closed form](https://mlsanity.com/paper/2306.03819/) (2023)
- [Adversarial Removal of Demographic Attributes from Text Data](https://mlsanity.com/paper/1808.06640/) (2018)
- [Towards Robust and Privacy-preserving Text Representations](https://mlsanity.com/paper/1805.06093/) (2018)
- [Text and Causal Inference: A Review of Using Text to Remove Confounding from Causal Estimates](https://mlsanity.com/paper/2005.00649/) (2020)
- [Privacy-preserving Neural Representations of Text](https://mlsanity.com/paper/1808.09408/) (2018)
- [Gradient Reversal Against Discrimination](https://mlsanity.com/paper/1807.00392/) (2018)
- [Better Hit the Nail on the Head than Beat around the Bush: Removing Protected Attributes with a Single Projection](https://mlsanity.com/paper/2212.04273/) (2022)
- [Causal Estimation for Text Data with (Apparent) Overlap Violations](https://mlsanity.com/paper/2210.00079/) (2022)
- [Conceptualizing Treatment Leakage in Text-based Causal Inference](https://mlsanity.com/paper/2205.00465/) (2022)

## Abstract (arXiv)

We investigate the behavior of methods that use linear projections to remove information about a concept from a language representation, and we consider the question of what happens to a dataset transformed by such a method. A theoretical analysis and experiments on real-world and synthetic data show that these methods inject strong statistical dependencies into the transformed datasets. After applying such a method, the representation space is highly structured: in the transformed space, an instance tends to be located near instances of the opposite label. As a consequence, the original labeling can in some cases be reconstructed by applying an anti-clustering method.