# The Low-Dimensional Linear Geometry of Contextualized Word Representations

arXiv: https://arxiv.org/abs/2105.07109 · HTML: https://mlsanity.com/paper/2105.07109/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 40 references that are in the map.
- First posted on arXiv 15 May 2021.

## Builds on (most cited first, 28 of 28)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [BERT Rediscovers the Classical NLP Pipeline](https://mlsanity.com/paper/1905.05950/) (2019)
- [How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings](https://mlsanity.com/paper/1909.00512/) (2019)
- [Network Dissection: Quantifying Interpretability of Deep Visual Representations](https://mlsanity.com/paper/1704.05796/) (2017)
- [Designing and Interpreting Probes with Control Tasks](https://mlsanity.com/paper/1909.03368/) (2019)
- [What you can cram into a single vector: Probing sentence embeddings for linguistic properties](https://mlsanity.com/paper/1805.01070/) (2018)
- [Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection](https://mlsanity.com/paper/2004.07667/) (2020)
- [What do you learn from context? Probing for sentence structure in contextualized word representations](https://mlsanity.com/paper/1905.06316/) (2019)
- [Linguistic Knowledge and Transferability of Contextual Representations](https://mlsanity.com/paper/1903.08855/) (2019)
- [Analysis Methods in Neural Language Processing: A Survey](https://mlsanity.com/paper/1812.08951/) (2018)
- [Targeted Syntactic Evaluation of Language Models](https://mlsanity.com/paper/1808.09031/) (2018)
- [Information-Theoretic Probing with Minimum Description Length](https://mlsanity.com/paper/2003.12298/) (2020)
- [Visualizing and Measuring the Geometry of BERT](https://mlsanity.com/paper/1906.02715/) (2019)
- [Information-Theoretic Probing for Linguistic Structure](https://mlsanity.com/paper/2004.03061/) (2020)
- [To Tune or Not to Tune? Adapting Pretrained Representations to Diverse Tasks](https://mlsanity.com/paper/1903.05987/) (2019)
- [Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance?](https://mlsanity.com/paper/2005.00719/) (2020)
- [Learning to Generate Reviews and Discovering Sentiment](https://mlsanity.com/paper/1704.01444/) (2017)
- [The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives](https://mlsanity.com/paper/1909.01380/) (2019)
- [Neural Language Models as Psycholinguistic Subjects: Representations of Syntactic State](https://mlsanity.com/paper/1903.03260/) (2019)
- [What Is One Grain of Sand in the Desert? Analyzing Individual Neurons in Deep NLP Models](https://mlsanity.com/paper/1812.09355/) (2018)
- [Identifying and Controlling Important Neurons in Neural Machine Translation](https://mlsanity.com/paper/1811.01157/) (2018)
- [Pareto Probing: Trading Off Accuracy for Complexity](https://mlsanity.com/paper/2010.02180/) (2020)
- [Exploring the Linear Subspace Hypothesis in Gender Bias Mitigation](https://mlsanity.com/paper/2009.09435/) (2020)
- [Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals](https://mlsanity.com/paper/2006.00995/) (2020)

## Built on by (most cited first, 30 of 40)

- [Refusal in Language Models Is Mediated by a Single Direction](https://mlsanity.com/paper/2406.11717/) (2024)
- [Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task](https://mlsanity.com/paper/2210.13382/) (2022)
- [LEACE: Perfect linear concept erasure in closed form](https://mlsanity.com/paper/2306.03819/) (2023)
- [Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Color](https://mlsanity.com/paper/2109.06129/) (2021)
- [Linear Adversarial Concept Erasure](https://mlsanity.com/paper/2201.12091/) (2022)
- [BrainBERT: Self-supervised representation learning for intracranial recordings](https://mlsanity.com/paper/2302.14367/) (2023)
- [Cognitive Dissonance: Why Do Language Model Outputs Disagree with Internal Representations of Truthfulness?](https://mlsanity.com/paper/2312.03729/) (2023)
- [Kernelized Concept Erasure](https://mlsanity.com/paper/2201.12191/) (2022)
- [Coloring the Blank Slate: Pre-training Imparts a Hierarchical Inductive Bias to Sequence-to-sequence Models](https://mlsanity.com/paper/2203.09397/) (2022)
- [Bridging Information-Theoretic and Geometric Compression in Language Models](https://mlsanity.com/paper/2310.13620/) (2023)
- [Mechanistic?](https://mlsanity.com/paper/2410.09087/) (2024)
- [COSMIC: Generalized Refusal Direction Identification in LLM Activations](https://mlsanity.com/paper/2506.00085/) (2025)
- [Uncovering hidden geometry in Transformers via disentangling position and context](https://mlsanity.com/paper/2310.04861/) (2023)
- [Robust AI-Generated Text Detection by Restricted Embeddings](https://mlsanity.com/paper/2410.08113/) (2024)
- [Geometric Signatures of Compositionality Across a Language Model's Lifetime](https://mlsanity.com/paper/2410.01444/) (2024)
- [Outlier Dimensions Encode Task-Specific Knowledge](https://mlsanity.com/paper/2310.17715/) (2023)
- [Orthogonality and isotropy of speaker and phonetic information in self-supervised speech representations](https://mlsanity.com/paper/2406.09200/) (2024)
- [Characterizing Large Language Model Geometry Helps Solve Toxicity Detection and Generation](https://mlsanity.com/paper/2312.01648/) (2023)
- [RepIt: Steering Language Models with Concept-Specific Refusal Vectors](https://mlsanity.com/paper/2509.13281/) (2025)
- [Putting Words in BERT's Mouth: Navigating Contextualized Vector Spaces with Pseudowords](https://mlsanity.com/paper/2109.11491/) (2021)
- [The Role of Interactive Visualization in Explaining (Large) NLP Models: from Data to Inference](https://mlsanity.com/paper/2301.04528/) (2023)
- [Brittle Minds, Fixable Activations: Understanding Belief Representations in Language Models](https://mlsanity.com/paper/2406.17513/) (2024)
- [Linguistic Interpretability of Transformer-based Language Models: a systematic review](https://mlsanity.com/paper/2504.08001/) (2025)
- [Transformer Normalisation Layers and the Independence of Semantic Subspaces](https://mlsanity.com/paper/2406.17837/) (2024)
- [Designing Role Vectors to Improve LLM Inference Behaviour](https://mlsanity.com/paper/2502.12055/) (2025)
- [Visual Instruction Bottleneck Tuning](https://mlsanity.com/paper/2505.13946/) (2025)
- [Reasoning in Large Language Models: A Geometric Perspective](https://mlsanity.com/paper/2407.02678/) (2024)
- [AutoML-guided Fusion of Entity and LLM-based Representations for Document Classification](https://mlsanity.com/paper/2408.09794/) (2024)
- [Analyzing the relationships between pretraining language, phonetic, tonal, and speaker information in self-supervised speech models](https://mlsanity.com/paper/2506.10855/) (2025)
- [Toward Preference-aligned Large Language Models via Residual-based Model Steering](https://mlsanity.com/paper/2509.23982/) (2025)

## Abstract (arXiv)

Black-box probing models can reliably extract linguistic features like tense, number, and syntactic role from pretrained word representations. However, the manner in which these features are encoded in representations remains poorly understood. We present a systematic study of the linear geometry of contextualized word representations in ELMO and BERT. We show that a variety of linguistic features (including structured dependency relationships) are encoded in low-dimensional subspaces. We then refine this geometric picture, showing that there are hierarchical relations between the subspaces encoding general linguistic categories and more specific ones, and that low-dimensional feature encodings are distributed rather than aligned to individual neurons. Finally, we demonstrate that these linear subspaces are causally related to model behavior, and can be used to perform fine-grained manipulation of BERT's output distribution.