# Group Crosscoders for Mechanistic Analysis of Symmetry

arXiv: https://arxiv.org/abs/2410.24184 · HTML: https://mlsanity.com/paper/2410.24184/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Sparse autoencoder training and evaluation, one of 13 directions in Mechanistic interpretability.
- Builds on 8 of its 18 references that are in the map.
- First posted on arXiv 31 Oct 2024.

## Placed in

- Mechanistic interpretability / [Sparse autoencoder training and evaluation](https://mlsanity.com/mechinterp/sparse-autoencoder-training-and-evaluation/)

## Builds on (most cited first, 8 of 8)

- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Toy Models of Superposition](https://mlsanity.com/paper/2209.10652/) (2022)
- [Understanding image representations by measuring their equivariance and equivalence](https://mlsanity.com/paper/1411.5908/) (2014)
- [Linear Algebraic Structure of Word Senses, with Applications to Polysemy](https://mlsanity.com/paper/1601.03764/) (2016)
- [Exploiting Cyclic Symmetry in Convolutional Neural Networks](https://mlsanity.com/paper/1602.02660/) (2016)
- [The Missing Curve Detectors of InceptionV1: Applying Sparse Autoencoders to InceptionV1 Early Vision](https://mlsanity.com/paper/2406.03662/) (2024)

## Built on by (most cited first, 3 of 3)

- [Understanding sparse autoencoder scaling in the presence of feature manifolds](https://mlsanity.com/paper/2509.02565/) (2025)
- [Naturally Computed Scale Invariance in the Residual Stream of ResNet18](https://mlsanity.com/paper/2504.16290/) (2025)
- [From Interpretability Methods to Interpretable Models](https://mlsanity.com/paper/2609.05399/) (2026)

## Abstract (arXiv)

We introduce group crosscoders, an extension of crosscoders that systematically discover and analyse symmetrical features in neural networks. While neural networks often develop equivariant representations without explicit architectural constraints, understanding these emergent symmetries has traditionally relied on manual analysis. Group crosscoders automate this process by performing dictionary learning across transformed versions of inputs under a symmetry group. Applied to InceptionV1's mixed3b layer using the dihedral group D32, our method reveals several key insights: First, it naturally clusters features into interpretable families that correspond to previously hypothesised feature types, providing more precise separation than standard sparse autoencoders. Second, our transform block analysis enables the automatic characterisation of feature symmetries, revealing how different geometric features (such as curves versus lines) exhibit distinct patterns of invariance and equivariance. These results demonstrate that group crosscoders can provide systematic insights into how neural networks represent symmetry, offering a promising new tool for mechanistic interpretability.