mlsanityData to 7 Oct 2026

Linearity of Relation Decoding in Transformer Language Models

arXiv 2308.09124 ↗First posted 17 Aug 2023cs.CLcited by 190

Not on the map. Why?

Abstract (arXiv)

Much of the knowledge encoded in transformer language models (LMs) may be expressed in terms of relations: relations between words and their synonyms, entities and their attributes, etc. We show that, for a subset of relations, this computation is well-approximated by a single linear transformation on the subject representation. Linear relation representations may be obtained by constructing a first-order approximation to the LM from a single prompt, and they exist for a variety of factual, commonsense, and linguistic relations. However, we also identify many cases in which LM predictions capture relational knowledge accurately, but this knowledge is not linearly encoded in their representations. Our results thus reveal a simple, interpretable, but heterogeneously deployed knowledge representation strategy in transformer LMs.

Key facts. Cited by 190 papers in the map (data through 7 Oct 2026). Builds on 28 of its 49 references that are in the map. First posted on arXiv 17 Aug 2023.

Builds on

28 of 49 references.

References · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2016Layer Normalizationcited by 5,579
  5. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  6. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  7. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  8. 2019Language Models as Knowledge Bases?cited by 1,922
  9. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  10. 2019How Can We Know What Language Models Know?cited by 953
  11. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  12. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
16 more
  1. 2020A Survey on Knowledge Graphs: Representation, Acquisition and Applicationscited by 643
  2. 2019Designing and Interpreting Probes with Control Taskscited by 608
  3. 2021Knowledge Neurons in Pretrained Transformerscited by 590
  4. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
  5. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
  6. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 512
  7. 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 452
  8. 2018Analysis Methods in Neural Language Processing: A Surveycited by 276
  9. 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 237
  10. 2020Information-Theoretic Probing with Minimum Description Lengthcited by 230
  11. 2021Implicit Representations of Meaning in Neural Language Modelscited by 179
  12. 2017What do Neural Machine Translation Models Learn about Morphology?cited by 165
  13. 2023Inspecting and Editing Knowledge Representations in Language Modelscited by 140
  14. 2023Language Models Implement Simple Word2Vec-style Vector Arithmeticcited by 97
  15. 2023Jump to Conclusions: Short-Cutting Transformers With Linear Transformationscited by 89
  16. 2023Crawling the Internal Knowledge-Base of Language Modelscited by 85

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 10; 2024: 54; 2025: 50; 2026: 76.

190 papers.

Sort:

By area:

  1. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
  2. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  3. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  4. 2023Language Models Represent Space and Timecited by 306
  5. 2023Function Vectors in Large Language Modelscited by 292
  6. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 225
  7. 2024A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicitycited by 179
  8. 2024Do Large Language Models Latently Perform Multi-Hop Reasoning?cited by 165
  9. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 157
  10. 2023Future Lens: Anticipating Subsequent Tokens from a Single Hidden Statecited by 116
  11. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 113
  12. 2023How do Language Models Bind Entities in Context?cited by 102
38 more
  1. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 83
  2. 2024RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representationscited by 77
  3. 2023Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Modelscited by 67
  4. 2024Attention Heads of Large Language Models: A Surveycited by 65
  5. 2024SelfIE: Self-Interpretation of Large Language Model Embeddingscited by 63
  6. 2024Knowledge Circuits in Pretrained Transformerscited by 60
  7. 2024LatentQA: Teaching LLMs to Decode Activations Into Natural Languagecited by 49
  8. 2024Do language models plan ahead for future tokens?cited by 47
  9. 2025From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuitcited by 45
  10. 2024On Large Language Models' Hallucination with Regard to Known Factscited by 43
  11. 2025Lost in Multilinguality: Dissecting Cross-lingual Factual Inconsistency in Transformer Language Modelscited by 43
  12. 2025Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainerscited by 41
  13. 2024Who's asking? User personas and the mechanics of latent misalignmentcited by 40
  14. 2024Summing Up the Facts: Additive Mechanisms Behind Factual Recall in LLMscited by 39
  15. 2024Estimating Knowledge in Large Language Models Without Generating a Single Tokencited by 39
  16. 2024Monitoring Latent World States in Language Models with Propositional Probescited by 39
  17. 2024Mechanistic Understanding and Mitigation of Language Model Non-Factual Hallucinationscited by 38
  18. 2024Locating and Editing Factual Associations in Mambacited by 35
  19. 2025Same Task, Different Circuits: Disentangling Modality-Specific Mechanisms in VLMscited by 35
  20. 2024Safety-Aware Fine-Tuning of Large Language Modelscited by 33
  21. 2024Towards Unifying Interpretability and Control: Evaluation via Interventioncited by 28
  22. 2024Representational Analysis of Binding in Language Modelscited by 27
  23. 2024Relational Composition in Neural Networks: A Survey and Call to Actioncited by 23
  24. 2025On Linear Representations and Pretraining Data Frequency in Language Modelscited by 23
  25. 2025The Dual-Route Model of Inductioncited by 22
  26. 2024Optimal ablation for interpretabilitycited by 21
  27. 2024Position: Understanding LLMs Requires More Than Statistical Generalizationcited by 20
  28. 2024Residual Stream Analysis with Multi-Layer SAEscited by 18
  29. 2024InstructEdit: Instruction-based Knowledge Editing for Large Language Modelscited by 17
  30. 2024Inferring Functionality of Attention Heads from their Parameterscited by 17
  31. 2024In-Context Learning State Vector with Inner and Momentum Optimizationcited by 15
  32. 2025Communicating Activations Between Language Model Agentscited by 15
  33. 2025How Do LLMs Use Their Depth?cited by 15
  34. 2026Subliminal Effects in Your Data: A General Mechanism via Log-Linearitycited by 15
  35. 2024Evidence from counterfactual tasks supports emergent analogical reasoning in large language modelscited by 14
  36. 2024All or None: Identifiable Linear Properties of Next-token Predictors in Language Modelingcited by 14
  37. 2025Does Time Have Its Place? Temporal Heads: Where Language Models Recall Time-specific Informationcited by 14
  38. 2024Poser: Unmasking Alignment Faking LLMs by Manipulating Their Internalscited by 13

50 most cited of 190 All 190 (JSON)