mlsanityData to 14 Oct 2026

The Low-Dimensional Linear Geometry of Contextualized Word Representations

arXiv 2105.07109 ↗First posted 15 May 2021cs.CLcited by 40

Not on the map. Why?

Abstract (arXiv)

Black-box probing models can reliably extract linguistic features like tense, number, and syntactic role from pretrained word representations. However, the manner in which these features are encoded in representations remains poorly understood. We present a systematic study of the linear geometry of contextualized word representations in ELMO and BERT. We show that a variety of linguistic features (including structured dependency relationships) are encoded in low-dimensional subspaces. We then refine this geometric picture, showing that there are hierarchical relations between the subspaces encoding general linguistic categories and more specific ones, and that low-dimensional feature encodings are distributed rather than aligned to individual neurons. Finally, we demonstrate that these linear subspaces are causally related to model behavior, and can be used to perform fine-grained manipulation of BERT's output distribution.

Key facts. Cited by 40 papers in the map (data through 14 Oct 2026). Builds on 28 of its 40 references that are in the map. First posted on arXiv 15 May 2021.

Builds on

28 of 40 references.

References · 28

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2018Deep contextualized word representationscited by 2,290
  4. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  5. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
  6. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,059
  7. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 784
  8. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 708
  9. 2019Designing and Interpreting Probes with Control Taskscited by 640
  10. 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 457
  11. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 455
  12. 2019What do you learn from context? Probing for sentence structure in contextualized word representationscited by 438
16 more
  1. 2019Linguistic Knowledge and Transferability of Contextual Representationscited by 383
  2. 2018Analysis Methods in Neural Language Processing: A Surveycited by 277
  3. 2018Targeted Syntactic Evaluation of Language Modelscited by 242
  4. 2020Information-Theoretic Probing with Minimum Description Lengthcited by 232
  5. 2019Visualizing and Measuring the Geometry of BERTcited by 216
  6. 2020Information-Theoretic Probing for Linguistic Structurecited by 207
  7. 2019To Tune or Not to Tune? Adapting Pretrained Representations to Diverse Taskscited by 167
  8. 2020Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance?cited by 151
  9. 2017Learning to Generate Reviews and Discovering Sentimentcited by 148
  10. 2019The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectivescited by 139
  11. 2019Neural Language Models as Psycholinguistic Subjects: Representations of Syntactic Statecited by 138
  12. 2018What Is One Grain of Sand in the Desert? Analyzing Individual Neurons in Deep NLP Modelscited by 124
  13. 2018Identifying and Controlling Important Neurons in Neural Machine Translationcited by 111
  14. 2020Pareto Probing: Trading Off Accuracy for Complexitycited by 55
  15. 2020Exploring the Linear Subspace Hypothesis in Gender Bias Mitigationcited by 25
  16. 2020Amnesic Probing: Behavioral Explanation with Amnesic Counterfactualscited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 5; 2023: 10; 2024: 9; 2025: 9; 2026: 5.

40 papers.

Sort:

By area:

  1. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  2. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 431
  3. 2023LEACE: Perfect linear concept erasure in closed formcited by 267
  4. 2021Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Colorcited by 119
  5. 2022Linear Adversarial Concept Erasurecited by 101
  6. 2023BrainBERT: Self-supervised representation learning for intracranial recordingscited by 72
  7. 2023Cognitive Dissonance: Why Do Language Model Outputs Disagree with Internal Representations of Truthfulness?cited by 62
  8. 2022Kernelized Concept Erasurecited by 48
  9. 2022Coloring the Blank Slate: Pre-training Imparts a Hierarchical Inductive Bias to Sequence-to-sequence Modelscited by 28
  10. 2023Bridging Information-Theoretic and Geometric Compression in Language Modelscited by 26
  11. 2024Mechanistic?cited by 25
  12. 2025COSMIC: Generalized Refusal Direction Identification in LLM Activationscited by 19
28 more
  1. 2023Uncovering hidden geometry in Transformers via disentangling position and contextcited by 18
  2. 2024Robust AI-Generated Text Detection by Restricted Embeddingscited by 18
  3. 2024Geometric Signatures of Compositionality Across a Language Model's Lifetimecited by 16
  4. 2023Outlier Dimensions Encode Task-Specific Knowledgecited by 15
  5. 2024Orthogonality and isotropy of speaker and phonetic information in self-supervised speech representationscited by 14
  6. 2023Characterizing Large Language Model Geometry Helps Solve Toxicity Detection and Generationcited by 12
  7. 2025RepIt: Steering Language Models with Concept-Specific Refusal Vectorscited by 12
  8. 2021Putting Words in BERT's Mouth: Navigating Contextualized Vector Spaces with Pseudowordscited by 9
  9. 2023The Role of Interactive Visualization in Explaining (Large) NLP Models: from Data to Inferencecited by 9
  10. 2024Brittle Minds, Fixable Activations: Understanding Belief Representations in Language Modelscited by 8
  11. 2025Linguistic Interpretability of Transformer-based Language Models: a systematic reviewcited by 7
  12. 2024Transformer Normalisation Layers and the Independence of Semantic Subspacescited by 6
  13. 2025Designing Role Vectors to Improve LLM Inference Behaviourcited by 6
  14. 2025Visual Instruction Bottleneck Tuningcited by 5
  15. 2024Reasoning in Large Language Models: A Geometric Perspectivecited by 4
  16. 2024AutoML-guided Fusion of Entity and LLM-based Representations for Document Classificationcited by 3
  17. 2025Analyzing the relationships between pretraining language, phonetic, tonal, and speaker information in self-supervised speech modelscited by 3
  18. 2025Toward Preference-aligned Large Language Models via Residual-based Model Steeringcited by 3
  19. 2026A framework for analyzing concept representations in neural modelscited by 2
  20. 2022Reprint: a randomized extrapolation based on principal components for data augmentationcited by 1
  21. 2023Syntax-guided Neural Module Distillation to Probe Compositionality in Sentence Embeddingscited by 1
  22. 2023Semantic Composition in Visually Grounded Language Modelscited by 1
  23. 2025Relative Overfitting and Accept-Reject Frameworkcited by 1
  24. 2025Emergent World Beliefs: Exploring Transformers in Stochastic Gamescited by 1
  25. 2026The Grammar of Transformers: A Systematic Review of Interpretability Research on Syntactic Knowledge in Language Modelscited by 1
  26. 2026Trait-space Monitoring for Emergent Misalignment During Supervised Finetuningcited by 0
  27. 2026Global Divergence, Local Convergence: Representation Geometry in SSMs and Transformerscited by 0
  28. 2026Deep Speech Representations Track Word Timing along a Low-Dimensional Circular Trajectorycited by 0