# Patches of Nonlinearity: Instruction Vectors in Large Language Models

arXiv: https://arxiv.org/abs/2602.07930 · HTML: https://mlsanity.com/paper/2602.07930/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 33 references that are in the map.
- First posted on arXiv 8 Feb 2026.

## Builds on (most cited first, 22 of 22)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [LIMA: Less Is More for Alignment](https://mlsanity.com/paper/2305.11206/) (2023)
- [Towards Automated Circuit Discovery for Mechanistic Interpretability](https://mlsanity.com/paper/2304.14997/) (2023)
- [Towards Best Practices of Activation Patching in Language Models: Metrics and Methods](https://mlsanity.com/paper/2309.16042/) (2023)
- [In-Context Learning Creates Task Vectors](https://mlsanity.com/paper/2310.15916/) (2023)
- [Deep ReLU Networks Have Surprisingly Few Activation Patterns](https://mlsanity.com/paper/1906.00904/) (2019)
- [Improving Instruction-Following in Language Models through Activation Steering](https://mlsanity.com/paper/2410.12877/) (2024)
- [A Closer Look at the Limitations of Instruction Tuning](https://mlsanity.com/paper/2402.05119/) (2024)
- [Chat Vector: A Simple Approach to Equip LLMs with Instruction Following and Model Alignment in New Languages](https://mlsanity.com/paper/2310.04799/) (2023)
- [NNsight and NDIF: Democratizing Access to Open-Weight Foundation Model Internals](https://mlsanity.com/paper/2407.14561/) (2024)
- [Instruction Tuning With Loss Over Instructions](https://mlsanity.com/paper/2405.14394/) (2024)
- [The Non-Linear Representation Dilemma: Is Causal Abstraction Enough for Mechanistic Interpretability?](https://mlsanity.com/paper/2507.08802/) (2025)
- [SAIF: A Sparse Autoencoder Framework for Interpreting and Steering Instruction Following of Language Models](https://mlsanity.com/paper/2502.11356/) (2025)
- [Interpreting Neural Networks through the Polytope Lens](https://mlsanity.com/paper/2211.12312/) (2022)
- [Just-in-time and distributed task representations in language models](https://mlsanity.com/paper/2509.04466/) (2025)
- [From Instance Training to Instruction Learning: Task Adapters Generation from Instructions](https://mlsanity.com/paper/2406.12382/) (2024)
- [Toward a Flexible Framework for Linear Representation Hypothesis Using Maximum Likelihood Estimation](https://mlsanity.com/paper/2502.16385/) (2025)
- [Localized Definitions and Distributed Reasoning: A Proof-of-Concept Mechanistic Interpretability Study via Activation Patching](https://mlsanity.com/paper/2504.02976/) (2025)

## Built on by (most cited first, 1 of 1)

- [Same Targets, Different Computation: How Post-Training Divides Work Across Model Layers](https://mlsanity.com/paper/2605.07284/) (2026)

## Abstract (arXiv)

Despite the recent success of instruction-tuned language models and their ubiquitous usage, very little is known of how models process instructions internally. In this work, we address this gap from a mechanistic point of view by investigating how instruction-specific representations are constructed and utilized in different stages of post-training: Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO). Via causal mediation, we identify that instruction representation is fairly localized in models. These representations, which we call Instruction Vectors (IVs), demonstrate a curious juxtaposition of linear separability along with non-linear causal interaction, broadly questioning the scope of the linear representation hypothesis commonplace in mechanistic interpretability. To disentangle the non-linear causal interaction, we propose a novel method to localize information processing in language models that is free from the implicit linear assumptions of patching-based techniques. We find that, conditioned on the task representations formed in the early layers, different information pathways are selected in the later layers to solve that task, i.e., IVs act as circuit selectors.