# Causality neq Invariance: Function and Concept Vectors in LLMs

arXiv: https://arxiv.org/abs/2602.22424 · HTML: https://mlsanity.com/paper/2602.22424/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 33 references that are in the map.
- First posted on arXiv 25 Feb 2026.

## Builds on (most cited first, 21 of 21)

- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [The Linear Representation Hypothesis and the Geometry of Large Language Models](https://mlsanity.com/paper/2311.03658/) (2023)
- [In-Context Learning Creates Task Vectors](https://mlsanity.com/paper/2310.15916/) (2023)
- [Function Vectors in Large Language Models](https://mlsanity.com/paper/2310.15213/) (2023)
- [Linearity of Relation Decoding in Transformer Language Models](https://mlsanity.com/paper/2308.09124/) (2023)
- [The Hydra Effect: Emergent Self-repair in Language Model Computations](https://mlsanity.com/paper/2307.15771/) (2023)
- [How do Language Models Bind Entities in Context?](https://mlsanity.com/paper/2310.17191/) (2023)
- [Attention Heads of Large Language Models: A Survey](https://mlsanity.com/paper/2409.03752/) (2024)
- [Which Attention Heads Matter for In-Context Learning?](https://mlsanity.com/paper/2502.14010/) (2025)
- [Learning to Make Analogies by Contrasting Abstract Relational Structure](https://mlsanity.com/paper/1902.00120/) (2019)
- [Identifying Semantic Induction Heads to Understand In-Context Learning](https://mlsanity.com/paper/2402.13055/) (2024)
- [Answer, Assemble, Ace: Understanding How LMs Answer Multiple Choice Questions](https://mlsanity.com/paper/2407.15018/) (2024)
- [Emergent Symbolic Mechanisms Support Abstract Reasoning in Large Language Models](https://mlsanity.com/paper/2502.20332/) (2025)
- [Human-like object concept representations emerge naturally in multimodal large language models](https://mlsanity.com/paper/2407.01067/) (2024)
- [On Linear Representations and Pretraining Data Frequency in Language Models](https://mlsanity.com/paper/2504.12459/) (2025)
- [Comparing Bottom-Up and Top-Down Steering Approaches on In-Context Learning Tasks](https://mlsanity.com/paper/2411.07213/) (2024)
- [Do different prompting methods yield a common task representation in language models?](https://mlsanity.com/paper/2505.12075/) (2025)
- [Contextualize-then-Aggregate: Circuits for In-Context Learning in Gemma-2 2B](https://mlsanity.com/paper/2504.00132/) (2025)
- [Emergence and Effectiveness of Task Vectors in In-Context Learning: An Encoder Decoder Perspective](https://mlsanity.com/paper/2412.12276/) (2024)
- [Provable In-Context Vector Arithmetic via Retrieving Task Concepts](https://mlsanity.com/paper/2508.09820/) (2025)
- [Whither symbols in the era of advanced neural networks?](https://mlsanity.com/paper/2508.05776/) (2025)

## Built on by (most cited first, 8 of 8)

- [Test, then Route: How Language Models Execute In-Context Conditional Rules Across Models and Languages](https://mlsanity.com/paper/2608.04183/) (2026)
- [Readable Before Actionable: Causal Tracing of Indirect Prompt Injection](https://mlsanity.com/paper/2610.05295/) (2026)
- [Necessary, Decodable and Reversible, Yet Not Transferable: A Stress Test for Attention-Head Role Claims](https://mlsanity.com/paper/2606.08292/) (2026)
- [A Pre-Specified Construction-Confirmation Test of Operation-Level Causal Transfer Across Finite Isomorphic Symbolic Domains](https://mlsanity.com/paper/2608.15809/) (2026)
- [How Do Language Models Choose Between Context and Memory?](https://mlsanity.com/paper/2609.00753/) (2026)
- [In-Context Learning Amplifies a Latent Symbolic Circuit](https://mlsanity.com/paper/2609.36265/) (2026)
- [Which Attention Heads are like the Human Head? Not the Ones that Compute](https://mlsanity.com/paper/2609.37991/) (2026)
- [The Geometry of Contextual Relations: Language Models Address Facts by Order of Mention](https://mlsanity.com/paper/2610.00910/) (2026)

## Abstract (arXiv)

Do large language models (LLMs) represent concepts abstractly, i.e., independent of input format? We revisit Function Vectors (FVs), compact representations of in-context learning (ICL) tasks that causally drive task performance. Across multiple LLMs, we show that FVs are not fully invariant: FVs are nearly orthogonal when extracted from different input formats (e.g., open-ended vs. multiple-choice), even if both target the same concept. We identify Concept Vectors (CVs), which carry more stable concept representations. Like FVs, CVs are composed of attention head outputs; however, unlike FVs, the constituent heads are selected using Representational Similarity Analysis (RSA) based on whether they encode concepts consistently across input formats. While these heads emerge in similar layers to FV-related heads, the two sets are largely distinct, suggesting different underlying mechanisms. Steering experiments reveal that FVs excel in-distribution, when extraction and application formats match (e.g., both open-ended in English), while CVs generalize better out-of-distribution across both question types (open-ended vs. multiple-choice) and languages. Our results show that LLMs do contain abstract concept representations, but these differ from those that drive ICL performance.