# Activation-Space Order-Swap Geometry: A Site-Asymmetry Audit

arXiv: https://arxiv.org/abs/2608.25315 · HTML: https://mlsanity.com/paper/2608.25315/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 34 references that are in the map.
- First posted on arXiv 26 Aug 2026.

## Builds on (most cited first, 28 of 28)

- [Editing Models with Task Arithmetic](https://mlsanity.com/paper/2212.04089/) (2022)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained Models](https://mlsanity.com/paper/2305.12827/) (2023)
- [The Shapley Taylor Interaction Index](https://mlsanity.com/paper/1902.05622/) (2019)
- [Analyzing the Generalization and Reliability of Steering Vectors](https://mlsanity.com/paper/2407.12404/) (2024)
- [Explaining Explanations: Axiomatic Feature Interactions for Deep Networks](https://mlsanity.com/paper/2002.04138/) (2020)
- [Detecting Statistical Interactions from Neural Network Weights](https://mlsanity.com/paper/1705.04977/) (2017)
- [Faith-Shap: The Faithful Shapley Interaction Index](https://mlsanity.com/paper/2203.00870/) (2022)
- [Extending Activation Steering to Broad Skills and Multiple Behaviours](https://mlsanity.com/paper/2403.05767/) (2024)
- [Automated Interpretability Metrics Do Not Distinguish Trained and Random Transformers](https://mlsanity.com/paper/2501.17727/) (2025)
- [The Curse of Multiple Mediators: Hidden Interaction Effects in Activation Patching](https://mlsanity.com/paper/2606.27510/) (2026)
- [Local Linearity of LLMs Enables Activation Steering via Model-Based Linear Optimal Control](https://mlsanity.com/paper/2604.19018/) (2026)
- [The Geometry of Sequential Learning: Lie-Bracket Prediction of Transfer Order](https://mlsanity.com/paper/2606.24993/) (2026)
- [Gauge-invariant representation holonomy](https://mlsanity.com/paper/2601.21653/) (2026)
- [Composable Interventions for Language Models](https://mlsanity.com/paper/2407.06483/) (2024)
- [First-Order Predictable but Pairwise Fragile: Local Task Adaptation in Trained Transformers](https://mlsanity.com/paper/2607.16821/) (2026)
- [Weight Updates as Activation Shifts: A Principled Framework for Steering](https://mlsanity.com/paper/2603.00425/) (2026)
- [When Attribution Patching Lies: Diagnosis and a Second-Order Correction](https://mlsanity.com/paper/2606.09899/) (2026)
- [A Low-Rank Subspace Analysis of LLM Interventions](https://mlsanity.com/paper/2606.14388/) (2026)
- [Infinitesimal Causality](https://mlsanity.com/paper/2606.24621/) (2026)
- [When Is a Steerable Concept Representation Real? Measurement Confounds in a Cross-Family Audit of Neuroscience Parallels in LLMs](https://mlsanity.com/paper/2608.08159/) (2026)
- [Bilinear autoencoders find interpretable manifolds](https://mlsanity.com/paper/2605.08891/) (2026)
- [Do Active SAE Feature Planes Carry More Holonomy? A Preregistered Reversal in Gemma](https://mlsanity.com/paper/2607.20522/) (2026)
- [Forgetting Is Not a Fix: Path Dependence in Sequential Engram Editing](https://mlsanity.com/paper/2607.24805/) (2026)
- [Feed-Forward Steering in Transformer Residual Dynamics](https://mlsanity.com/paper/2608.02071/) (2026)
- [Cross-Layer Interaction under Weight-Space Ablation: A Closed-Form Attention Jacobian Bound and a Test on a Real Pretrained Model](https://mlsanity.com/paper/2608.03629/) (2026)
- [CircuitSteer: Geometrically Aligned Multi-Layer Steering via Sparse Autoencoder Circuits](https://mlsanity.com/paper/2608.05732/) (2026)

## Abstract (arXiv)

Order-dependent activation statistics are often interpreted as evidence of interaction, but that interpretation can be confounded by where interventions enter the network. We introduce a no-fit site-asymmetry audit. For a twice-differentiable readout, the open-path order-swap decomposes into a canonical additive response measured by single interventions and an antisymmetrized second difference free of first-order and pure self-curvature terms to second order. Across six open-weight language-model families, the single-intervention baseline explains 84.3-97.7 percent of the bracket norm (mean 93.7 percent), while the no-interaction self-curvature term is 1.8-5.2 times larger than the corrected residual in the two families with the plus/minus injection split. The corrected residual clears a generic-interaction null in three of six families under a confound-free prompt split and two of six after configuration robustness. A known-positive surrogate recovers planted mixed interaction, while a matched site-separation test changes the baseline share and a random architecture reproduces the first-order regime. The same estimator transfers to released non-language references: trained residual fractions fall below a fixed Gaussian-direction null in 11/12 contrasts (5/6 ViT-B/16, 6/6 ResNet-50), a portability check rather than pooled evidence. The contribution is a reusable measurement criterion: run the single-intervention baseline before reading an order-swap vector as interaction or geometric structure; if it explains the vector, form the second difference instead. All claims are scoped to activation-space interventions at distinct sites; we do not claim that representation geometry is globally Abelian.