Polymorphism Is Rotation: Operational Mechanistic Interpretability from a Two-Layer Transformer to Pythia-70m
On the map: Mechanistic interpretability → Circuits and mechanistic analysis
Abstract (arXiv)
Independently trained transformers compute the same function in residual-stream bases that differ by a uniform random rotation on SO(dmodel). We call this phenomenon polymorphism: same function, mutually unintelligible interior coordinates. One matrix multiplication per model pair removes it: an orthogonal Procrustes fit on a single batch of activations transfers sparse-autoencoder feature dictionaries and steering vectors between independently trained models, with no retraining. The phenomenon is invisible to the standard SAE universality metric. Decoder-column cosine similarity matches across seeds at 98%, the SAE-universality headline number, while an SAE trained on one seed reconstructs another seed's activations at negative explained variance, worse than predicting the constant mean. The decoder columns align; the encoder reads from a rotated frame. A single Procrustes rotation R restores reconstruction to within 0.025 EV of the within-seed ceiling at every internal site. R is Haar-distributed: |R - I|F matches the random-orthogonal prediction √2 dmodel to 0.1% at dmodel = 512, and a Kolmogorov-Smirnov test of R's eigenvalue spectrum against Haar SO(dmodel) returns p ≈ 1.000 pooled and per-pair. Diff-of-means steering vectors transfer in three regimes by alignment with R's invariant subspace: clean when pinned by shared output weights, partial when overlapping the rotated subspace, inverted otherwise. With no shared I/O (Pythia), all three collapse to universally inverted. The same rotation account holds across training checkpoints within a single run. Validated on a 104k-parameter Dyck-3 transformer and nine independently-trained Pythia-70m seeds on The Pile, via a pre-registered four-bar operational framework. Frontier-scale (10B+) replication remains open.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Circuits and mechanistic analysis, one of 13 directions in Mechanistic interpretability. Builds on 29 of its 40 references that are in the map. First posted on arXiv 23 May 2026.
Builds on
29 of 40 references.
Closest ancestors · 4
Same direction, newest first
- 2024Transcoders Find Interpretable LLM Feature Circuits17 Jun
- 2023Towards Automated Circuit Discovery for Mechanistic Interpretability28 Apr
- 2023Progress measures for grokking via mechanistic interpretability12 Jan
- 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small1 Nov
Other references · 25
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
- 2017Axiomatic Attribution for Deep Networkscited by 3,384
- 2022Locating and Editing Factual Associations in GPTcited by 2,691
- 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
- 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
- 2019Similarity of Neural Network Representations Revisitedcited by 1,755
- 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
- 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
- 2022Mass-Editing Memory in a Transformercited by 1,001
- 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
- 2019Designing and Interpreting Probes with Control Taskscited by 640
- 2022Git Re-Basin: Merging Models modulo Permutation Symmetriescited by 471
13 more
- 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 435
- 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 394
- 2023Function Vectors in Large Language Modelscited by 308
- 2021The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networkscited by 291
- 2019Model Fusion via Optimal Transportcited by 219
- 2023Localizing Model Behavior with Path Patchingcited by 195
- 2018Riemannian Adaptive Optimization Methodscited by 194
- 2021Thinking Like Transformerscited by 184
- 2021Generalized Shape Metrics on Neural Representationscited by 112
- 2024AtP*: An efficient and scalable method for localizing LLM behaviour to componentscited by 111
- 2023Tracr: Compiled Transformers as a Laboratory for Interpretabilitycited by 104
- 2020Optimizing Mode Connectivity via Neuron Alignmentcited by 86
- 2025Transferring Linear Features Across Language Models With Model Stitchingcited by 21
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Seed-Induced Uniqueness in Transformer Models: Subspace Alignment Governs Subliminal Transfer
- 2026On the Intrinsic Limits of Transformer Image Embeddings in Non-Solvable Spatial Reasoning
- 2026Symmetry Breaking in Transformers for Efficient and Interpretable Training
- 2026Causal Dimensionality of Transformer Representations: Measurement, Scaling, and Layer Structure
- 2024Interpreting Affine Recurrence Learning in GPT-style Transformers
- 2025Equivariant Sparse Autoencoders: Mechanistic Interpretability of Neural Networks on Symmetric Data
- 2025Beyond Components: Singular Vector-Based Interpretability of Transformer Circuits
- 2026Fixed Universal Transformers