Do Active SAE Feature Planes Carry More Holonomy? A Preregistered Reversal in Gemma
On the map: Mechanistic interpretability → Sparse autoencoder training and evaluation
Abstract (arXiv)
This paper tests whether holonomy concentrates on active sparse-autoencoder (SAE) feature planes in Gemma 2 2B, a concrete operationalization of the broader semantic-concentration prediction. Holonomy is measured at the final-token layer-12 to layer-13 residual-stream readout by carrying a local frame around small loops using the instrument's restricted-Jacobian transport rule, then normalizing the resulting rotation by enclosed area. The design, materiality threshold, analysis, and verdict rules were preregistered and frozen before the analysed measurements were inspected. The prediction was falsified in reverse: active-feature planes carried less holonomy than matched mixed-feature controls, with an adjusted log contrast of -0.29439 and 95% interval [-0.43989, -0.14889]. A magnitude-only explanation was not supported in this design, while the three-way ordering across random, mixed-feature, and active-feature planes was undefined at matched magnitude because common support failed. Post-freeze diagnostics at the same readout supported the area law on a small validation subset, bounded matched-center displacement under a simple paired regression, and identified transport distortion as a live mechanism or confound. The result is therefore a narrow, auditable operational reversal, not a causal claim that meaning suppresses holonomy. The cause remains open, with activation-strength geometry, degree of feature engagement, dictionary geometry, matched-center displacement, activation-manifold proximity, and transport shear as live alternatives.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Sparse autoencoder training and evaluation, one of 13 directions in Mechanistic interpretability. Builds on 6 of its 7 references that are in the map. First posted on arXiv 9 Jul 2026.
Builds on
6 of 7 references.
Closest ancestors · 2
Same direction, newest first
- 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 29 Aug
- 2022Toy Models of Superposition21 Sep
Other references · 4
- 2017Attention Is All You Needcited by 60,361
- 2016Pointer Sentinel Mixture Modelscited by 3,096
- 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
- 2026Gauge-invariant representation holonomycited by 7
Built on by
Citing papers per year: 2026: 1.
1 paper.
- 2026Activation-Space Order-Swap Geometry: A Site-Asymmetry Audit26 Aug 2026
Similar, not cited
Close in wording, no citation either way
- 2026From Geometric Recovery to Causal Validation: A Reproducible Audit of Sparse Autoencoder Features, from Superposition Geometry to Causal Inertness
- 2025Features Emerge as Discrete States: The First Application of SAEs to 3D Representations
- 2026A Dominant Diffuse Phase in the Sparse Autoencoder Phase Diagram
- 2026Active Budget Can Kill Sensitivity: Diagnosing and Repairing TopK Sparse Autoencoder Reliability
- 2025Causal Interpretation of Sparse Autoencoder Features in Vision
- 2026Steering grids for sparse-autoencoder features: when a top-context label names an activation regime rather than a causal axis
- 2026The Geometric Wall: Manifold Structure Predicts Layerwise Sparse Autoencoder Scaling Laws
- 2026Pre-Intervention Prediction of Sparse Autoencoder Steering Side Effects