Universal Response and Emergence of Induction in LLMs
Not on the map. Why?
Abstract (arXiv)
While induction is considered a key mechanism for in-context learning in LLMs, understanding its precise circuit decomposition beyond toy models remains elusive. Here, we study the emergence of induction behavior within LLMs by probing their response to weak single-token perturbations of the residual stream. We find that LLMs exhibit a robust, universal regime in which their response remains scale-invariant under changes in perturbation strength, thereby allowing us to quantify the build-up of token correlations throughout the model. By applying our method, we observe signatures of induction behavior within the residual stream of Gemma-2-2B, Llama-3.2-3B, and GPT-2-XL. Across all models, we find that these induction signatures gradually emerge within intermediate layers and identify the relevant model sections composing this behavior. Our results provide insights into the collective interplay of components within LLMs and serve as a benchmark for large-scale circuit analysis.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 41 of its 76 references that are in the map. First posted on arXiv 11 Nov 2024.
Builds on
41 of 76 references.
References · 41
- 2017Attention Is All You Needcited by 59,932
- 2023GPT-4 Technical Reportcited by 18,132
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
- 2022Locating and Editing Factual Associations in GPTcited by 2,609
- 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
- 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
- 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
- 2019Attention is not Explanationcited by 916
- 2023Progress measures for grokking via mechanistic interpretabilitycited by 822
- 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
- 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
29 more
- 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 512
- 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
- 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 412
- 2023Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representationscited by 197
- 2023Attribution Patching Outperforms Automated Circuit Discoverycited by 196
- 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 194
- 2023Localizing Model Behavior with Path Patchingcited by 182
- 2023The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networkscited by 153
- 2023Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchillacited by 145
- 2023A Toy Model of Universality: Reverse Engineering How Networks Learn Group Operationscited by 141
- 2024The Remarkable Robustness of LLMs: Stages of Inference?cited by 138
- 2022Analyzing Transformers in Embedding Spacecited by 131
- 2023Future Lens: Anticipating Subsequent Tokens from a Single Hidden Statecited by 116
- 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 113
- 2021Inducing Causal Structure for Interpretable Neural Networkscited by 107
- 2023Circuit Component Reuse Across Tasks in Transformer Language Modelscited by 105
- 2024Universal Neurons in GPT2 Language Modelscited by 100
- 2024Information Flow Routes: Automatically Interpreting Language Models at Scalecited by 92
- 2023Jump to Conclusions: Short-Cutting Transformers With Linear Transformationscited by 89
- 2024What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formationcited by 86
- 2023Successor Heads: Recurring, Interpretable Attention Heads In The Wildcited by 77
- 2023Copy Suppression: Comprehensively Understanding an Attention Headcited by 73
- 2019Attention Interpretability Across NLP Taskscited by 71
- 2024LLM Circuit Analyses Are Consistent Across Training and Scalecited by 60
- 2024Induction Heads as an Essential Mechanism for Pattern Matching in In-context Learningcited by 46
- 2024Unveiling Induction Heads: Provable Training Dynamics and Feature Learning in Transformerscited by 46
- 2023DecoderLens: Layerwise Interpretation of Encoder-Decoder Transformerscited by 22
- 2023Attention Lens: A Tool for Mechanistically Interpreting the Attention Head Information Retrieval Mechanismcited by 21
- 2024Investigating Representation Universality: Case Study on Genealogical Representationscited by 4
Built on by
Citing papers per year: 2025: 1.
1 paper.