Cross-Trajectory Chimera Interventions Reveal Dissociable Roles of Weight Magnitude and Direction in Grokking
Not on the map. Why?
Abstract (arXiv)
Which properties of a partially trained network are causally portable to a different, independently trained network? Single-trajectory interventions show necessity within one run, not portability across runs. We introduce cross-trajectory chimera interventions: given two runs from different seeds, we split each weight vector into a norm and a unit direction, recombine one run's norm with the other's direction, and continue training. On two modular-arithmetic tasks that grok, the components dissociate. Direction carries a transferable, donor-specific circuit identity: implanting a donor's direction at the recipient's norm drives the run to the donor's circuit in 40/40 cases, while an angle-matched random control yields no shift. The transfer is threshold-like, and its location is predicted by the recipient's norm, separating perfectly by norm class over all 20 pairs (joint permutation probability 1.9e-4). Norm carries only a modest, distributed delay effect and no identity signal. An adaptive bisection procedure localizes the threshold to +/-1/64. Direction indexes which solution a trajectory approaches; norm governs how susceptible that identity is to being overwritten.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 9 of its 12 references that are in the map. First posted on arXiv 7 Jul 2026.
Builds on
9 of 12 references.
References · 9
- 2023Progress measures for grokking via mechanistic interpretabilitycited by 839
- 2019Linear Mode Connectivity and the Lottery Ticket Hypothesiscited by 637
- 2022Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetscited by 629
- 2022Git Re-Basin: Merging Models modulo Permutation Symmetriescited by 471
- 2019Deep Ensembles: A Loss Landscape Perspectivecited by 443
- 2021The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networkscited by 291
- 2022Towards Understanding Grokking: An Effective Theory of Representation Learningcited by 230
- 2023Explaining grokking through circuit efficiencycited by 111
- 2026A Basin-Selection Perspective on Grokking via Singular Learning Theorycited by 4
Built on by
Not cited yet.