mlsanityData to 7 Oct 2026

Language Models Implement Simple Word2Vec-style Vector Arithmetic

arXiv 2305.16130 ↗First posted 25 May 2023cs.CL cs.LGcited by 97

Not on the map. Why?

Abstract (arXiv)

A primary criticism towards language models (LMs) is their inscrutability. This paper presents evidence that, despite their size and complexity, LMs sometimes exploit a simple vector arithmetic style mechanism to solve some relational tasks using regularities encoded in the hidden space of the model (e.g., Poland:Warsaw::China:Beijing). We investigate a range of language model sizes (from 124M parameters to 176B parameters) in an in-context learning setting, and find that for a variety of tasks (involving capital cities, uppercasing, and past-tensing) a key part of the mechanism reduces to a simple additive update typically applied by the feedforward (FFN) networks. We further show that this mechanism is specific to tasks that require retrieval from pretraining memory, rather than retrieval from local context. Our results contribute to a growing body of work on the interpretability of LMs, and offer reason to be optimistic that, despite the massive and non-linear nature of the models, the strategies they ultimately use to solve tasks can sometimes reduce to familiar and even intuitive algorithms.

Key facts. Cited by 97 papers in the map (data through 7 Oct 2026). Builds on 29 of its 39 references that are in the map. First posted on arXiv 25 May 2023.

Builds on

29 of 39 references.

References · 29

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  4. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  5. 2018Deep contextualized word representationscited by 2,285
  6. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  7. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  8. 2019Language Models as Knowledge Bases?cited by 1,922
  9. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  10. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  11. 2022Editing Models with Task Arithmeticcited by 1,076
  12. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
17 more
  1. 2022Mass-Editing Memory in a Transformercited by 976
  2. 2022In-context Learning and Induction Headscited by 930
  3. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  4. 2021Knowledge Neurons in Pretrained Transformerscited by 590
  5. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
  6. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 512
  7. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 417
  8. 2019The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectivescited by 139
  9. 2021Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Basescited by 107
  10. 2019Inducing Relational Knowledge from BERTcited by 75
  11. 2021When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Datacited by 75
  12. 2017Residual Connections Encourage Iterative Inferencecited by 74
  13. 2022Rethinking the Role of Scale for In-Context Learning: An Interpretability-based Case Study at 66 Billion Scalecited by 64
  14. 2022Discovering Latent Concepts Learned in BERTcited by 56
  15. 2020Towards falsifiable interpretability researchcited by 53
  16. 2021Analyzing Commonsense Emergence in Few-shot Knowledge Modelscited by 34
  17. 2021Static Embeddings as Efficient Knowledge Bases?cited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 19; 2024: 18; 2025: 39; 2026: 21.

97 papers.

Sort:

By area:

  1. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
  2. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 328
  3. 2023In-Context Learning Creates Task Vectorscited by 302
  4. 2023Function Vectors in Large Language Modelscited by 292
  5. 2023Linearity of Relation Decoding in Transformer Language Modelscited by 190
  6. 2023Circuit Component Reuse Across Tasks in Transformer Language Modelscited by 105
  7. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 83
  8. 2024Hopping Too Late: Exploring the Limitations of Large Language Models on Multi-Hop Queriescited by 78
  9. 2023Successor Heads: Recurring, Interpretable Attention Heads In The Wildcited by 77
  10. 2024The Semantic Hub Hypothesis: Language Models Share Semantic Representations Across Languages and Modalitiescited by 69
  11. 2024Knowledge Circuits in Pretrained Transformerscited by 60
  12. 2023Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Modelscited by 50
38 more
  1. 2024From Tokens to Words: On the Inner Lexicon of LLMscited by 46
  2. 2023Codebook Features: Sparse and Discrete Interpretability for Neural Networkscited by 45
  3. 2025From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuitcited by 45
  4. 2023Rigorously Assessing Natural Language Explanations of Neuronscited by 44
  5. 2024Pre-trained Large Language Models Use Fourier Features to Compute Additioncited by 38
  6. 2023Finding and Editing Multi-Modal Neurons in Pre-Trained Transformerscited by 36
  7. 2024Locate-then-edit for Multi-hop Factual Recall under Knowledge Editingcited by 36
  8. 2024Answer, Assemble, Ace: Understanding How LMs Answer Multiple Choice Questionscited by 35
  9. 2025M2IV: Towards Efficient and Fine-grained Multimodal In-Context Learning via Representation Engineeringcited by 34
  10. 2024The Same But Different: Structural Similarities and Differences in Multilingual Language Modelingcited by 27
  11. 2023LLMs as Hackers: Autonomous Linux Privilege Escalation Attackscited by 25
  12. 2025On Linear Representations and Pretraining Data Frequency in Language Modelscited by 23
  13. 2023DecoderLens: Layerwise Interpretation of Encoder-Decoder Transformerscited by 22
  14. 2024Optimal ablation for interpretabilitycited by 21
  15. 2024Position: Understanding LLMs Requires More Than Statistical Generalizationcited by 20
  16. 2025Paths Not Taken: Understanding and Mending the Multilingual Factual Recall Pipelinecited by 20
  17. 2025BiasEdit: Debiasing Stereotyped Language Models via Model Editingcited by 18
  18. 2025AudioLens: A Closer Look at Auditory Attribute Perception of Large Audio-Language Modelscited by 18
  19. 2023Uncovering Meanings of Embeddings via Partial Orthogonalitycited by 15
  20. 2024In-Context Learning State Vector with Inner and Momentum Optimizationcited by 15
  21. 2025How Do LLMs Use Their Depth?cited by 15
  22. 2024All or None: Identifiable Linear Properties of Next-token Predictors in Language Modelingcited by 14
  23. 2023Uncovering Intermediate Variables in Transformers using Circuit Probingcited by 12
  24. 2025Is Random Attention Sufficient for Sequence Modeling? Disentangling Trainable Components in the Transformercited by 12
  25. 2024Position: An Inner Interpretability Framework for AI Inspired by Lessons from Cognitive Neurosciencecited by 11
  26. 2025Learning Task Representations from In-Context Learningcited by 11
  27. 2025SkipGPT: Dynamic Layer Pruning Reinvented with Token Awareness and Module Decouplingcited by 11
  28. 2023NeuroSurgeon: A Toolkit for Subnetwork Analysiscited by 9
  29. 2024Emergence and Effectiveness of Task Vectors in In-Context Learning: An Encoder Decoder Perspectivecited by 9
  30. 2025CaKE: Circuit-aware Editing Enables Generalizable Knowledge Learnerscited by 9
  31. 2025On the Emergence of Linear Analogies in Word Embeddingscited by 9
  32. 2025Can LLMs Reason Abstractly Over Math Word Problems Without CoT? Disentangling Abstract Formulation From Arithmetic Computationcited by 8
  33. 2025Train Once, Reuse Everywhere: Generalizable Implicit In-Context Learning by Routing Attentioncited by 8
  34. 2024ELICIT: LLM Augmentation via External In-Context Capabilitycited by 7
  35. 2025How Do Language Models Compose Functions?cited by 7
  36. 2023Instilling Inductive Biases with Subnetworkscited by 6
  37. 2024When Parts Are Greater Than Sums: Individual LLM Components Can Outperform Full Modelscited by 6
  38. 2025Sparse Feature Coactivation Reveals Causal Semantic Modules in Large Language Modelscited by 6

50 most cited of 97 All 97 (JSON)