# A Unified Framework for Vision Transformers Equivariant to Discrete Subgroups of O(2)

arXiv: https://arxiv.org/abs/2606.27864 · HTML: https://mlsanity.com/paper/2606.27864/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 36 of its 43 references that are in the map.
- First posted on arXiv 26 Jun 2026.

## Builds on (most cited first, 30 of 36)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [VGGT: Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2503.11651/) (2025)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [E(n) Equivariant Graph Neural Networks](https://mlsanity.com/paper/2102.09844/) (2021)
- [E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials](https://mlsanity.com/paper/2101.03164/) (2021)
- [SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks](https://mlsanity.com/paper/2006.10503/) (2020)
- [MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields](https://mlsanity.com/paper/2206.07697/) (2022)
- [General E(2)-Equivariant Steerable CNNs](https://mlsanity.com/paper/1911.08251/) (2019)
- [Rotation Equivariant CNNs for Digital Pathology](https://mlsanity.com/paper/1806.03962/) (2018)
- [3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data](https://mlsanity.com/paper/1807.02547/) (2018)
- [Invariant and Equivariant Graph Networks](https://mlsanity.com/paper/1812.09902/) (2018)
- [Steerable CNNs](https://mlsanity.com/paper/1612.08498/) (2016)
- [On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups](https://mlsanity.com/paper/1802.03690/) (2018)
- [Cormorant: Covariant Molecular Neural Networks](https://mlsanity.com/paper/1906.04015/) (2019)
- [A General Theory of Equivariant CNNs on Homogeneous Spaces](https://mlsanity.com/paper/1811.02017/) (2018)
- [Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Network](https://mlsanity.com/paper/1806.09231/) (2018)
- [Artificial Intelligence for Science in Quantum, Atomistic, and Continuum Systems](https://mlsanity.com/paper/2307.08423/) (2023)
- [Equivariance with Learned Canonicalization Functions](https://mlsanity.com/paper/2211.06489/) (2022)
- [Roto-Translation Covariant Convolutional Networks for Medical Image Analysis](https://mlsanity.com/paper/1804.03393/) (2018)
- [Geometric Deep Learning and Equivariant Neural Networks](https://mlsanity.com/paper/2105.13926/) (2021)
- [On the Symmetries of Deep Learning Models and their Internal Representations](https://mlsanity.com/paper/2205.14258/) (2022)
- [Universal Equivariant Multilayer Perceptrons](https://mlsanity.com/paper/2002.02912/) (2020)
- [Equivariant Transformer Networks](https://mlsanity.com/paper/1901.11399/) (2019)
- [SE(3)-Equivariant Attention Networks for Shape Reconstruction in Function Space](https://mlsanity.com/paper/2204.02394/) (2022)
- [VN-Transformer: Rotation-Equivariant Attention for Vector Neurons](https://mlsanity.com/paper/2206.04176/) (2022)
- [E(2)-Equivariant Vision Transformer](https://mlsanity.com/paper/2306.06722/) (2023)
- [E(n) Equivariant Message Passing Simplicial Networks](https://mlsanity.com/paper/2305.07100/) (2023)
- [Platonic Transformers: A Solid Choice For Equivariance](https://mlsanity.com/paper/2510.03511/) (2025)

## Abstract (arXiv)

Vision transformers have become a dominant architecture for visual recognition. However, standard models do not explicitly encode the planar symmetries that arise in many vision domains. We introduce a family of vision transformers equivariant to arbitrary discrete subgroups of O(2), providing a unified framework that generalizes prior flipping- and D4-equivariant transformer architectures. Our construction yields equivariant analogues of the core transformer components, together with expressivity guarantees for the resulting layers. In particular, we show that whenever H ≤ G, the class of G-equivariant ViTs embeds naturally into the class of H-equivariant ViTs. We also prove that, in the single-head setting, the corresponding equivariant self-attention layer realizes every G-equivariant self-attention map representable by ordinary self-attention. We further construct a D6-equivariant model based on hexagonal patches, making the architecture compatible with six-fold rotational symmetries. We evaluate the resulting models on the PatternNet aerial image dataset in artificially data-scarce regimes across subgroups of D4 and D6. Our experiments compare two equivariant attention mechanisms and analyze how the choice of homogeneous-space configurations used in the nonlinearities affects performance. Preliminary results under matched parameter budgets indicate that equivariance can improve recognition accuracy, motivating further study of how discrete symmetry groups shape transformer-based visual recognition models.