# Kernelized Concept Erasure

arXiv: https://arxiv.org/abs/2201.12191 · HTML: https://mlsanity.com/paper/2201.12191/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 51 references that are in the map.
- First posted on arXiv 28 Jan 2022.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks](https://mlsanity.com/paper/1908.02265/) (2019)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection](https://mlsanity.com/paper/2004.07667/) (2020)
- [Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting](https://mlsanity.com/paper/1901.09451/) (2019)
- [Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them](https://mlsanity.com/paper/1903.03862/) (2019)
- [SimLex-999: Evaluating Semantic Models with (Genuine) Similarity Estimation](https://mlsanity.com/paper/1408.3456/) (2014)
- [It's All in the Name: Mitigating Gender Bias with Name-Based Counterfactual Data Substitution](https://mlsanity.com/paper/1909.00871/) (2019)
- [Multimodal Pretraining Unmasked: A Meta-Analysis and a Unified Framework of Vision-and-Language BERTs](https://mlsanity.com/paper/2011.15124/) (2020)
- [Attenuating Bias in Word Vectors](https://mlsanity.com/paper/1901.07656/) (2019)
- [Counterfactual Interventions Reveal the Causal Effect of Relative Clause Representations on Agreement Prediction](https://mlsanity.com/paper/2105.06965/) (2021)
- [The Low-Dimensional Linear Geometry of Contextualized Word Representations](https://mlsanity.com/paper/2105.07109/) (2021)
- [A Non-Linear Structural Probe](https://mlsanity.com/paper/2105.10185/) (2021)
- [It's not Greek to mBERT: Inducing Word-Level Translations from Multilingual BERT](https://mlsanity.com/paper/2010.08275/) (2020)
- [Exploring the Linear Subspace Hypothesis in Gender Bias Mitigation](https://mlsanity.com/paper/2009.09435/) (2020)
- [On the Global Optima of Kernelized Adversarial Representation Learning](https://mlsanity.com/paper/1910.07423/) (2019)

## Built on by (most cited first, 30 of 48)

- [Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models](https://mlsanity.com/paper/2403.19647/) (2024)
- [LEACE: Perfect linear concept erasure in closed form](https://mlsanity.com/paper/2306.03819/) (2023)
- [In-Context Unlearning: Language Models as Few Shot Unlearners](https://mlsanity.com/paper/2310.07579/) (2023)
- [Emergent and Predictable Memorization in Large Language Models](https://mlsanity.com/paper/2304.11158/) (2023)
- [Probing for the Usage of Grammatical Number](https://mlsanity.com/paper/2204.08831/) (2022)
- [Self-Destructing Models: Increasing the Costs of Harmful Dual Uses of Foundation Models](https://mlsanity.com/paper/2211.14946/) (2022)
- [Representation Surgery: Theory and Practice of Affine Steering](https://mlsanity.com/paper/2402.09631/) (2024)
- [Probing Classifiers are Unreliable for Concept Removal and Detection](https://mlsanity.com/paper/2207.04153/) (2022)
- [Steering Out-of-Distribution Generalization with Concept Ablation Fine-Tuning](https://mlsanity.com/paper/2507.16795/) (2025)
- [Machine Unlearning Fails to Remove Data Poisoning Attacks](https://mlsanity.com/paper/2406.17216/) (2024)
- [Exploring Safety-Utility Trade-Offs in Personalized Language Models](https://mlsanity.com/paper/2406.11107/) (2024)
- [Shielded Representations: Protecting Sensitive Attributes Through Iterative Gradient-Based Projection](https://mlsanity.com/paper/2305.10204/) (2023)
- [The Quest for the Right Mediator: Surveying Mechanistic Interpretability Through the Lens of Causal Mediation Analysis](https://mlsanity.com/paper/2408.01416/) (2024)
- [Naturalistic Causal Probing for Morpho-Syntax](https://mlsanity.com/paper/2205.07043/) (2022)
- [Gumbel Counterfactual Generation From Language Models](https://mlsanity.com/paper/2411.07180/) (2024)
- [Robust Concept Erasure via Kernelized Rate-Distortion Maximization](https://mlsanity.com/paper/2312.00194/) (2023)
- [How Reliable are Causal Probing Interventions?](https://mlsanity.com/paper/2408.15510/) (2024)
- [Opt-Out: Investigating Entity-Level Unlearning for Large Language Models via Optimal Transport](https://mlsanity.com/paper/2406.12329/) (2024)
- [Protecting Privacy Through Approximating Optimal Parameters for Sequence Unlearning in Language Models](https://mlsanity.com/paper/2406.14091/) (2024)
- [Can Prompts Rewind Time for LLMs? Evaluating the Effectiveness of Prompted Knowledge Cutoffs](https://mlsanity.com/paper/2510.02340/) (2025)
- [Gen-Z: Generative Zero-Shot Text Classification with Contextualized Label Descriptions](https://mlsanity.com/paper/2311.07115/) (2023)
- [Memory in Large Language Models: Mechanisms, Evaluation and Evolution](https://mlsanity.com/paper/2509.18868/) (2025)
- [Removing Spurious Concepts from Neural Network Representations via Joint Subspace Estimation](https://mlsanity.com/paper/2310.11991/) (2023)
- [Fundamental Limits of Perfect Concept Erasure](https://mlsanity.com/paper/2503.20098/) (2025)
- [Fair Streaming Principal Component Analysis: Statistical and Algorithmic Viewpoint](https://mlsanity.com/paper/2310.18593/) (2023)
- [Focus On This, Not That! Steering LLMs with Adaptive Feature Specification](https://mlsanity.com/paper/2410.22944/) (2024)
- [Competence-Based Analysis of Language Models](https://mlsanity.com/paper/2303.00333/) (2023)
- [Improving Causal Interventions in Amnesic Probing with Mean Projection or LEACE](https://mlsanity.com/paper/2506.11673/) (2025)
- [Explaining Text Classifiers with Counterfactual Representations](https://mlsanity.com/paper/2402.00711/) (2024)
- [Leveraging Prototypical Representations for Mitigating Social Bias without Demographic Information](https://mlsanity.com/paper/2403.09516/) (2024)

## Abstract (arXiv)

The representation space of neural models for textual data emerges in an unsupervised manner during training. Understanding how those representations encode human-interpretable concepts is a fundamental problem. One prominent approach for the identification of concepts in neural representations is searching for a linear subspace whose erasure prevents the prediction of the concept from the representations. However, while many linear erasure algorithms are tractable and interpretable, neural networks do not necessarily represent concepts in a linear manner. To identify non-linearly encoded concepts, we propose a kernelization of a linear minimax game for concept erasure. We demonstrate that it is possible to prevent specific non-linear adversaries from predicting the concept. However, the protection does not transfer to different nonlinear adversaries. Therefore, exhaustively erasing a non-linearly encoded concept remains an open problem.