# Toward a Theory of Generalizability in LLM Mechanistic Interpretability Research arXiv: https://arxiv.org/abs/2509.22831 ยท HTML: https://mlsanity.com/paper/2509.22831/ ## Key facts - Cited by 5 papers in the map (data through 14 Oct 2026). - Placed in Knowledge and representations inside LLMs, one of 13 directions in Mechanistic interpretability. - Builds on 40 of its 78 references that are in the map. - First posted on arXiv 26 Sep 2025. ## Placed in - Mechanistic interpretability / [Knowledge and representations inside LLMs](https://mlsanity.com/mechinterp/knowledge-and-representations-inside-llms/) ## Builds on (most cited first, 30 of 40) - [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020) - [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019) - [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018) - [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023) - [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022) - [What Does BERT Look At? An Analysis of BERT's Attention](https://mlsanity.com/paper/1906.04341/) (2019) - [Towards Automated Circuit Discovery for Mechanistic Interpretability](https://mlsanity.com/paper/2304.14997/) (2023) - [The Platonic Representation Hypothesis](https://mlsanity.com/paper/2405.07987/) (2024) - [AI and the Everything in the Whole Wide World Benchmark](https://mlsanity.com/paper/2111.15366/) (2021) - [Finding Neurons in a Haystack: Case Studies with Sparse Probing](https://mlsanity.com/paper/2305.01610/) (2023) - [Convergent Learning: Do different neural networks learn the same representations?](https://mlsanity.com/paper/1511.07543/) (2015) - [Similarity of Neural Network Models: A Survey of Functional and Representational Measures](https://mlsanity.com/paper/2305.06329/) (2023) - [Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMs](https://mlsanity.com/paper/2309.07311/) (2023) - [Circuit Component Reuse Across Tasks in Transformer Language Models](https://mlsanity.com/paper/2310.08744/) (2023) - [What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation](https://mlsanity.com/paper/2404.07129/) (2024) - [LLM Circuit Analyses Are Consistent Across Training and Scale](https://mlsanity.com/paper/2407.10827/) (2024) - [Which Attention Heads Matter for In-Context Learning?](https://mlsanity.com/paper/2502.14010/) (2025) - [To Code, or Not To Code? Exploring Impact of Code in Pre-training](https://mlsanity.com/paper/2408.10914/) (2024) - [Modeling rapid language learning by distilling Bayesian priors into artificial neural networks](https://mlsanity.com/paper/2305.14701/) (2023) - [Talking Heads: Understanding Inter-layer Communication in Transformer Language Models](https://mlsanity.com/paper/2406.09519/) (2024) - [The Natural Stories Corpus](https://mlsanity.com/paper/1708.05763/) (2017) - [Between Circuits and Chomsky: Pre-pretraining on Formal Languages Imparts Linguistic Biases](https://mlsanity.com/paper/2502.19249/) (2025) - [Benchmarks as Microscopes: A Call for Model Metrology](https://mlsanity.com/paper/2407.16711/) (2024) - [PolyPythias: Stability and Outliers across Fifty Language Model Pre-Training Runs](https://mlsanity.com/paper/2503.09543/) (2025) - [When Is Multilinguality a Curse? Language Modeling for 250 High- and Low-Resource Languages](https://mlsanity.com/paper/2311.09205/) (2023) - [The Same But Different: Structural Similarities and Differences in Multilingual Language Modeling](https://mlsanity.com/paper/2410.09223/) (2024) - [The Dual-Route Model of Induction](https://mlsanity.com/paper/2504.03022/) (2025) - [How to Plant Trees in Language Models: Data and Architectural Effects on the Emergence of Syntactic Inductive Biases](https://mlsanity.com/paper/2305.19905/) (2023) - [PhyloLM : Inferring the Phylogeny of Large Language Models and Predicting their Performances in Benchmarks](https://mlsanity.com/paper/2404.04671/) (2024) - [Latent State Models of Training Dynamics](https://mlsanity.com/paper/2308.09543/) (2023) ## Built on by (most cited first, 5 of 5) - [Start Making Sense(s): A Developmental Probe of Attention Specialization Using Lexical Ambiguity](https://mlsanity.com/paper/2511.21974/) (2025) - [Differentiable Faithfulness Alignment for Cross-Model Circuit Transfer](https://mlsanity.com/paper/2604.24302/) (2026) - [Traces of Social Competence in Large Language Models](https://mlsanity.com/paper/2603.04161/) (2026) - [Developmental Trajectories of Situation Modeling and Mentalizing in Transformer Language Models](https://mlsanity.com/paper/2606.28524/) (2026) - [Shared Geometry Is Not Shared Physics: A Layerwise Test of the Platonic Representation Hypothesis in Astronomy](https://mlsanity.com/paper/2610.04130/) (2026) ## Abstract (arXiv) Research on Large Language Models (LLMs) increasingly focuses on identifying mechanistic explanations for their behaviors, yet the field lacks clear principles for determining when (and how) findings from one model instance generalize to another. This paper addresses a fundamental epistemological challenge: given a mechanistic claim about a particular model, what justifies extrapolating this finding to other LLMs -- and along which dimensions might such generalizations hold? I propose five potential axes of correspondence along which mechanistic claims might generalize, including: functional (whether they satisfy the same functional criteria), developmental (whether they develop at similar points during pretraining), positional (whether they occupy similar absolute or relative positions), relational (whether they interact with other model components in similar ways), and configurational (whether they correspond to particular regions or structures in weight-space). To empirically validate this framework, I analyze "1-back attention heads" (components attending to previous tokens) across pretraining in random seeds of the Pythia models (14M, 70M, 160M, 410M). The results reveal striking consistency in the developmental trajectories of 1-back attention across models, while positional consistency is more limited. Moreover, seeds of larger models systematically show earlier onsets, steeper slopes, and higher peaks of 1-back attention. I also address possible objections to the arguments and proposals outlined here. Finally, I conclude by arguing that progress on the generalizability of mechanistic interpretability research will consist in mapping constitutive design properties of LLMs to their emergent behaviors and mechanisms.