mlsanityData to 14 Oct 2026

The Linear Representation Hypothesis and the Geometry of Large Language Models

arXiv 2311.03658 ↗First posted 7 Nov 2023cs.CL cs.AI cs.LGcited by 716

Not on the map. Why?

Cited by 204 in Mechanistic interpretability, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Informally, the 'linear representation hypothesis' is the idea that high-level concepts are represented linearly as directions in some representation space. In this paper, we address two closely related questions: What does "linear representation" actually mean? And, how do we make sense of geometric notions (e.g., cosine similarity or projection) in the representation space? To answer these, we use the language of counterfactuals to give two formalizations of "linear representation", one in the output (word) representation space, and one in the input (sentence) space. We then prove these connect to linear probing and model steering, respectively. To make sense of geometric notions, we use the formalization to identify a particular (non-Euclidean) inner product that respects language structure in a sense we make precise. Using this causal inner product, we show how to unify all notions of linear representation. In particular, this allows the construction of probes and steering vectors using counterfactual pairs. Experiments with LLaMA-2 demonstrate the existence of linear representations of concepts, the connection to interpretation and control, and the fundamental role of the choice of inner product.

Key facts. Cited by 716 papers in the map (data through 14 Oct 2026). Builds on 43 of its 65 references that are in the map. First posted on arXiv 7 Nov 2023.

Builds on

43 of 65 references.

References · 43

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  5. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  6. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  7. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  8. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  9. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  10. 2022Toy Models of Superpositioncited by 971
  11. 2023Steering Language Models With Activation Engineeringcited by 922
  12. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
31 more
  1. 2020A Primer in BERTology: What we know about how BERT workscited by 815
  2. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 784
  3. 2015Generating Sentences from a Continuous Spacecited by 612
  4. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 532
  5. 2019Variational Autoencoders and Nonlinear ICA: A Unifying Frameworkcited by 503
  6. 2017Word Translation Without Parallel Datacited by 487
  7. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 431
  8. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 319
  9. 2023Language Models Represent Space and Timecited by 316
  10. 2023In-Context Learning Creates Task Vectorscited by 312
  11. 2023Function Vectors in Large Language Modelscited by 308
  12. 2018Towards a Definition of Disentangled Representationscited by 296
  13. 2016Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICAcited by 273
  14. 2021Contrastive Learning Inverts the Data Generating Processcited by 221
  15. 2019Visualizing and Measuring the Geometry of BERTcited by 216
  16. 2023Linearity of Relation Decoding in Transformer Language Modelscited by 200
  17. 2013Exploiting Similarities among Languages for Machine Translationcited by 191
  18. 2017A Survey Of Cross-lingual Word Embedding Modelscited by 143
  19. 2014word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding methodcited by 112
  20. 2023Language Models Implement Simple Word2Vec-style Vector Arithmeticcited by 98
  21. 2022The Geometry of Multilingual Language Model Representationscited by 95
  22. 2021BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language Models Identify Analogies?cited by 56
  23. 2021Identifiable Deep Generative Models via Sparse Decodingcited by 54
  24. 2021Probing BERT in Hyperbolic Spacescited by 50
  25. 2023Linear Spaces of Meanings: Compositional Structures in Vision-Language Modelscited by 48
  26. 2015Take and Took, Gaggle and Goose, Book and Read: Evaluating the Utility of Vector Differences for Lexical Relation Learningcited by 29
  27. 2020Analogies minus analogy test: measuring regularities in word embeddingscited by 17
  28. 2023Uncovering Meanings of Embeddings via Partial Orthogonalitycited by 15
  29. 2019word2word: A Collection of Bilingual Lexicons for 3,564 Language Pairscited by 7
  30. 2023Prompt Algebra for Task Compositioncited by 6
  31. 2020Understanding Linearity of Cross-Lingual Word Embedding Mappingscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 88; 2025: 203; 2026: 424.

716 papers.

Sort:

By area:

  1. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  2. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
  3. 2025Layer by Layer: Uncovering Hidden Representations in Language Modelscited by 311
  4. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 218
  5. 2024Not All Language Model Features Are One-Dimensionally Linearcited by 212
  6. 2024Programming Refusal with Conditional Activation Steeringcited by 206
  7. 2024ReFT: Representation Finetuning for Language Modelscited by 197
  8. 2024A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicitycited by 185
  9. 2024Improving Dictionary Learning with Gated Sparse Autoencoderscited by 179
  10. 2024A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoderscited by 160
  11. 2024Improving Instruction-Following in Language Models through Activation Steeringcited by 160
  12. 2024The Geometry of Categorical and Hierarchical Concepts in Large Language Modelscited by 159
38 more
  1. 2024Analyzing the Generalization and Reliability of Steering Vectorscited by 143
  2. 2024Interpreting CLIP with Sparse Linear Concept Embeddings (SpLiCE)cited by 119
  3. 2024Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Modelscited by 115
  4. 2025The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independencecited by 115
  5. 2024Automatically Interpreting Millions of Features in Large Language Modelscited by 113
  6. 2025LLM Social Simulations Are a Promising Research Methodcited by 99
  7. 2025Model Organisms for Emergent Misalignmentcited by 91
  8. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 85
  9. 2026Verbalizable Representations Form a Global Workspace in Language Modelscited by 75
  10. 2025Latent Collaboration in Multi-Agent Systemscited by 73
  11. 2025Convergent Linear Representations of Emergent Misalignmentcited by 71
  12. 2024Attention Heads of Large Language Models: A Surveycited by 67
  13. 2024Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimizationcited by 67
  14. 2024Robust LLM safeguarding via refusal feature adversarial trainingcited by 64
  15. 2024Semantics-Adaptive Activation Intervention for LLMs via Dynamic Steering Vectorscited by 63
  16. 2025JBShield: Defending Large Language Models from Jailbreak Attacks through Activated Concept Analysis and Manipulationcited by 54
  17. 2025Angular Steering: Behavior Control via Rotation in Activation Spacecited by 53
  18. 2024The Geometry of Concepts: Sparse Autoencoder Feature Structurecited by 52
  19. 2025Attention Sinks and Compression Valleys in LLMs are Two Sides of the Same Coincited by 52
  20. 2024Measuring Progress in Dictionary Learning for Language Model Interpretability with Board Game Modelscited by 51
  21. 2024ICLR: In-Context Learning of Representationscited by 50
  22. 2025From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuitcited by 49
  23. 2025AlphaSteer: Learning Refusal Steering with Principled Null-Space Constraintcited by 49
  24. 2025Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinationscited by 48
  25. 2024Decomposing The Dark Matter of Sparse Autoencoderscited by 45
  26. 2025I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoderscited by 45
  27. 2024Talking Heads: Understanding Inter-layer Communication in Transformer Language Modelscited by 41
  28. 2025LLM Unlearning via Neural Activation Redirectioncited by 41
  29. 2025The Hidden Dimensions of LLM Alignment: A Multi-Dimensional Analysis of Orthogonal Safety Directionscited by 41
  30. 2024Who's asking? User personas and the mechanics of latent misalignmentcited by 40
  31. 2024Efficient Dictionary Learning with Switch Sparse Autoencoderscited by 39
  32. 2025Steering Out-of-Distribution Generalization with Concept Ablation Fine-Tuningcited by 39
  33. 2024Causal Language Modeling Can Elicit Search and Reasoning Capabilities on Logic Puzzlescited by 38
  34. 2025No Answer Needed: Predicting LLM Answer Accuracy from Question-Only Linear Probescited by 38
  35. 2025Shared Global and Local Geometry of Language Model Embeddingscited by 36
  36. 2024Model Editing as a Robust and Denoised variant of DPO: A Case Study on Toxicitycited by 35
  37. 2024Emergence of Hidden Capabilities: Exploring Learning Dynamics in Concept Spacecited by 35
  38. 2025Interpretable Steering of Large Language Models with Feature Guided Activation Additionscited by 35

50 most cited of 716 All 716 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025The Origins of Representation Manifolds in Large Language Models
  2. 2026Linear Representations of Hierarchical Concepts in Language Models
  3. 2026Linear Probes Detect Task Format, Not Reasoning Mode in Language Model Hidden States
  4. 2026Beneath the Surface of Chains-of-Thought: A Mechanistic Interpretation of Reasoning Operations in LLMs
  5. 2026Context Is King: How In-Context Specification Shapes the Geometry of Concepts
  6. 2025Large Language Models Encode Semantics and Alignment in Linearly Separable Representations
  7. 2026Convergence Without Understanding: When Language Models Agree on Representations but Disagree on Reasoning
  8. 2026The Anatomy of a Truth Direction: Knowledge-Dependent Dimensionality, a Relational Law, and a Shared Category Geometry in Small Language Models