mlsanityData to 7 Oct 2026

Function Vectors in Large Language Models

arXiv 2310.15213 ↗First posted 23 Oct 2023cs.CL cs.LGcited by 292

Not on the map. Why?

Cited by 28 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We report the presence of a simple neural mechanism that represents an input-output function as a vector within autoregressive transformer language models (LMs). Using causal mediation analysis on a diverse range of in-context-learning (ICL) tasks, we find that a small number attention heads transport a compact representation of the demonstrated task, which we call a function vector (FV). FVs are robust to changes in context, i.e., they trigger execution of the task on inputs such as zero-shot and natural text settings that do not resemble the ICL contexts from which they are collected. We test FVs across a range of tasks, models, and layers and find strong causal effects across settings in middle layers. We investigate the internal structure of FVs and find while that they often contain information that encodes the output space of the function, this information alone is not sufficient to reconstruct an FV. Finally, we test semantic vector composition in FVs, and find that to some extent they can be summed to create vectors that trigger new complex tasks. Our findings show that compact, causal internal vector representations of function abstractions can be explicitly extracted from LLMs. Our code and data are available at https://functions.baulab.info.

Key facts. Cited by 292 papers in the map (data through 7 Oct 2026). Builds on 41 of its 58 references that are in the map. First posted on arXiv 23 Oct 2023.

Builds on

41 of 58 references.

References · 41

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  5. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  6. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  7. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  8. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  9. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  10. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  11. 2022Editing Models with Task Arithmeticcited by 1,076
  12. 2022Mass-Editing Memory in a Transformercited by 976
29 more
  1. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
  2. 2022Transformers learn in-context by gradient descentcited by 714
  3. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  4. 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 700
  5. 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 671
  6. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  7. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
  8. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
  9. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 512
  10. 2017Word Translation Without Parallel Datacited by 480
  11. 2023Larger language models do in-context learning differentlycited by 386
  12. 2023Learning to Compress Prompts with Gist Tokenscited by 379
  13. 2013Direct and Indirect Effectscited by 276
  14. 2023Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learningcited by 235
  15. 2023Transformers as Algorithms: Generalization and Stability in In-context Learningcited by 220
  16. 2023Linearity of Relation Decoding in Transformer Language Modelscited by 190
  17. 2022Instruction Induction: From Few Examples to Natural Language Task Descriptionscited by 152
  18. 2023What In-Context Learning "Learns" In-Context: Disentangling Task Recognition and Task Learningcited by 149
  19. 2022Analyzing Transformers in Embedding Spacecited by 131
  20. 2023The Learnability of In-Context Learningcited by 126
  21. 2022Ground-Truth Labels Matter: A Deeper Look into Input-Label Demonstrationscited by 116
  22. 2023What and How does In-Context Learning Learn? Bayesian Model Averaging, Parameterization, and Generalizationcited by 112
  23. 2023Task-Specific Skill Localization in Fine-tuned Language Modelscited by 104
  24. 2023Language Models Implement Simple Word2Vec-style Vector Arithmeticcited by 97
  25. 2023A Theory of Emergent In-Context Learning as Implicit Structure Inductioncited by 86
  26. 2023Overthinking the Truth: Understanding how Language Models Process False Demonstrationscited by 78
  27. 2022Finding Skill Neurons in Pre-trained Transformer-based Language Modelscited by 65
  28. 2017Distinguishing Antonyms and Synonyms in a Pattern-based Neural Networkcited by 19
  29. 2023Investigating the Learning Behaviour of In-context Learning: A Comparison with Supervised Learningcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 72; 2025: 89; 2026: 129.

292 papers.

Sort:

By area:

  1. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
  2. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  3. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 412
  4. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 381
  5. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 292
  6. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modificationscited by 220
  7. 2024How to use and interpret activation patchingcited by 191
  8. 2024A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicitycited by 179
  9. 2024Improving Instruction-Following in Language Models through Activation Steeringcited by 149
  10. 2024The Remarkable Robustness of LLMs: Stages of Inference?cited by 138
  11. 2024Universal Neurons in GPT2 Language Modelscited by 100
  12. 2024The Surprising Effectiveness of Test-Time Training for Few-Shot Learningcited by 88
38 more
  1. 2024The Semantic Hub Hypothesis: Language Models Share Semantic Representations Across Languages and Modalitiescited by 69
  2. 2024Attention Heads of Large Language Models: A Surveycited by 65
  3. 2024Knowledge Circuits in Pretrained Transformerscited by 60
  4. 2024Spectral Filters, Dark Signals, and Attention Sinkscited by 54
  5. 2024Active-Dormant Attention Heads: Mechanistically Demystifying Extreme-Token Phenomena in LLMscited by 54
  6. 2024The Geometry of Concepts: Sparse Autoencoder Feature Structurecited by 52
  7. 2024Measuring and Controlling Instruction (In)Stability in Language Model Dialogscited by 51
  8. 2024Multimodal Task Vectors Enable Many-Shot Multimodal In-Context Learningcited by 49
  9. 2024LatentQA: Teaching LLMs to Decode Activations Into Natural Languagecited by 49
  10. 2024ICLR: In-Context Learning of Representationscited by 49
  11. 2024From Tokens to Words: On the Inner Lexicon of LLMscited by 46
  12. 2024Is Mamba Capable of In-Context Learning?cited by 45
  13. 2024Separating Tongue from Thought: Activation Patching Reveals Language-Agnostic Concept Representations in Transformerscited by 42
  14. 2025Learning without training: The implicit dynamics of in-context learningcited by 41
  15. 2025Understanding (Un)Reliability of Steering Vectors in Language Modelscited by 39
  16. 2025When is Task Vector Provably Effective for Model Editing? A Generalization Analysis of Nonlinear Transformerscited by 38
  17. 2024From Redundancy to Relevance: Information Flow in LVLMs Across Reasoning Taskscited by 37
  18. 2024Locating and Editing Factual Associations in Mambacited by 35
  19. 2024Answer, Assemble, Ace: Understanding How LMs Answer Multiple Choice Questionscited by 35
  20. 2025Emergent Abilities in Large Language Models: A Surveycited by 34
  21. 2025M2IV: Towards Efficient and Fine-grained Multimodal In-Context Learning via Representation Engineeringcited by 34
  22. 2025Steering Out-of-Distribution Generalization with Concept Ablation Fine-Tuningcited by 34
  23. 2024Steering Large Language Models using Conceptors: Improving Addition-Based Activation Engineeringcited by 33
  24. 2024Towards Reliable Evaluation of Behavior Steering Interventions in LLMscited by 33
  25. 2024Towards Universality: Studying Mechanistic Similarity Across Language Model Architecturescited by 31
  26. 2025Emergent Symbolic Mechanisms Support Abstract Reasoning in Large Language Modelscited by 31
  27. 2024Language Models Represent Beliefs of Self and Otherscited by 29
  28. 2025Towards Understanding Distilled Reasoning Models: A Representational Approachcited by 26
  29. 2025To Steer or Not to Steer? Mechanistic Error Reduction with Abstention for Language Modelscited by 26
  30. 2024Do LLMs "know" internally when they follow instructions?cited by 25
  31. 2025The emergence of sparse attention: impact of data distribution and benefits of repetitioncited by 25
  32. 2024Finding Visual Task Vectorscited by 24
  33. 2024Can In-context Learning Really Generalize to Out-of-distribution Tasks?cited by 23
  34. 2025On Linear Representations and Pretraining Data Frequency in Language Modelscited by 23
  35. 2024Configurable Foundation Models: Building LLMs from a Modular Perspectivecited by 22
  36. 2024What Do Learning Dynamics Reveal About Generalization in LLM Reasoning?cited by 22
  37. 2025The Dual-Route Model of Inductioncited by 22
  38. 2026Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behaviorcited by 22

50 most cited of 292 All 292 (JSON)