Activation-Space Order-Swap Geometry: A Site-Asymmetry Audit
Not on the map. Why?
Abstract (arXiv)
Order-dependent activation statistics are often interpreted as evidence of interaction, but that interpretation can be confounded by where interventions enter the network. We introduce a no-fit site-asymmetry audit. For a twice-differentiable readout, the open-path order-swap decomposes into a canonical additive response measured by single interventions and an antisymmetrized second difference free of first-order and pure self-curvature terms to second order. Across six open-weight language-model families, the single-intervention baseline explains 84.3-97.7 percent of the bracket norm (mean 93.7 percent), while the no-interaction self-curvature term is 1.8-5.2 times larger than the corrected residual in the two families with the plus/minus injection split. The corrected residual clears a generic-interaction null in three of six families under a confound-free prompt split and two of six after configuration robustness. A known-positive surrogate recovers planted mixed interaction, while a matched site-separation test changes the baseline share and a random architecture reproduces the first-order regime. The same estimator transfers to released non-language references: trained residual fractions fall below a fixed Gaussian-direction null in 11/12 contrasts (5/6 ViT-B/16, 6/6 ResNet-50), a portability check rather than pooled evidence. The contribution is a reusable measurement criterion: run the single-intervention baseline before reading an order-swap vector as interaction or geometric structure; if it explains the vector, form the second difference instead. All claims are scoped to activation-space interventions at distinct sites; we do not claim that representation geometry is globally Abelian.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 28 of its 34 references that are in the map. First posted on arXiv 26 Aug 2026.
Builds on
28 of 34 references.
References · 28
- 2022Editing Models with Task Arithmeticcited by 1,120
- 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
- 2023Steering Language Models With Activation Engineeringcited by 922
- 2023Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained Modelscited by 221
- 2019The Shapley Taylor Interaction Indexcited by 143
- 2024Analyzing the Generalization and Reliability of Steering Vectorscited by 143
- 2020Explaining Explanations: Axiomatic Feature Interactions for Deep Networkscited by 103
- 2017Detecting Statistical Interactions from Neural Network Weightscited by 80
- 2022Faith-Shap: The Faithful Shapley Interaction Indexcited by 69
- 2024Extending Activation Steering to Broad Skills and Multiple Behaviourscited by 40
- 2025Automated Interpretability Metrics Do Not Distinguish Trained and Random Transformerscited by 30
- 2026The Curse of Multiple Mediators: Hidden Interaction Effects in Activation Patchingcited by 11
16 more
- 2026Local Linearity of LLMs Enables Activation Steering via Model-Based Linear Optimal Controlcited by 9
- 2026The Geometry of Sequential Learning: Lie-Bracket Prediction of Transfer Ordercited by 9
- 2026Gauge-invariant representation holonomycited by 7
- 2024Composable Interventions for Language Modelscited by 4
- 2026First-Order Predictable but Pairwise Fragile: Local Task Adaptation in Trained Transformerscited by 4
- 2026Weight Updates as Activation Shifts: A Principled Framework for Steeringcited by 3
- 2026When Attribution Patching Lies: Diagnosis and a Second-Order Correctioncited by 3
- 2026A Low-Rank Subspace Analysis of LLM Interventionscited by 3
- 2026Infinitesimal Causalitycited by 2
- 2026When Is a Steerable Concept Representation Real? Measurement Confounds in a Cross-Family Audit of Neuroscience Parallels in LLMscited by 2
- 2026Bilinear autoencoders find interpretable manifoldscited by 1
- 2026Do Active SAE Feature Planes Carry More Holonomy? A Preregistered Reversal in Gemmacited by 1
- 2026Forgetting Is Not a Fix: Path Dependence in Sequential Engram Editingcited by 1
- 2026Feed-Forward Steering in Transformer Residual Dynamicscited by 1
- 2026Cross-Layer Interaction under Weight-Space Ablation: A Closed-Form Attention Jacobian Bound and a Test on a Real Pretrained Modelcited by 1
- 2026CircuitSteer: Geometrically Aligned Multi-Layer Steering via Sparse Autoencoder Circuitscited by 1
Built on by
Not cited yet.