Harmformer: Harmonic Networks Meet Transformers for Continuous Roto-Translation Equivariance
Not on the map. Why?
Abstract (arXiv)
CNNs exhibit inherent equivariance to image translation, leading to efficient parameter and data usage, faster learning, and improved robustness. The concept of translation equivariant networks has been successfully extended to rotation transformation using group convolution for discrete rotation groups and harmonic functions for the continuous rotation group encompassing 360^circ. We explore the compatibility of the SA mechanism with full rotation equivariance, in contrast to previous studies that focused on discrete rotation. We introduce the Harmformer, a harmonic transformer with a convolutional stem that achieves equivariance for both translation and continuous rotation. Accompanied by an end-to-end equivariance proof, the Harmformer not only outperforms previous equivariant transformers, but also demonstrates inherent stability under any continuous rotation, even without seeing rotated samples during training.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 40 of its 53 references that are in the map. First posted on arXiv 6 Nov 2024.
Builds on
40 of 53 references.
References · 40
- 2017Attention Is All You Needcited by 60,361
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
- 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
- 2016Group Equivariant Convolutional Networkscited by 1,165
- 2020SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networkscited by 607
- 2021CoAtNet: Marrying Convolution and Attention for All Data Sizescited by 569
- 2019Stand-Alone Self-Attention in Vision Modelscited by 542
- 2018Spherical CNNscited by 456
- 2019General E(2)-Equivariant Steerable CNNscited by 446
- 2018Rotation Equivariant CNNs for Digital Pathologycited by 422
- 2021Early Convolutions Help Transformers See Bettercited by 396
28 more
- 20183D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Datacited by 354
- 2016Harmonic Networks: Deep Translation and Rotation Equivariancecited by 295
- 2022Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphscited by 264
- 2021Rethinking and Improving Relative Position Encoding for Vision Transformercited by 207
- 2023EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representationscited by 204
- 2021ReDet: A Rotation-equivariant Detector for Aerial Object Detectioncited by 203
- 2017Learning Steerable Filters for Rotation Equivariant CNNscited by 194
- 2022TorchMD-NET: Equivariant Transformers for Neural Network based Molecular Potentialscited by 182
- 2018Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Networkcited by 171
- 2022Equivariant 3D-Conditional Diffusion Models for Molecular Linker Designcited by 122
- 2021A Survey of Visual Transformerscited by 117
- 2019Deep Scale-spaces: Equivariance Over Scalecited by 114
- 2019Scale-Equivariant Steerable Networkscited by 110
- 2020LieTransformer: Equivariant self-attention for Lie Groupscited by 100
- 2018Roto-Translation Covariant Convolutional Networks for Medical Image Analysiscited by 92
- 2021Coordinate Independent Convolutional Networks -- Isometry and Gauge Equivariant Convolutions on Riemannian Manifoldscited by 73
- 2020Group Equivariant Stand-Alone Self-Attention For Visioncited by 63
- 2018CubeNet: Equivariance to 3D Rotation and Translationcited by 59
- 2023Fast, Expressive SE(n) Equivariant Networks through Weight-Sharing in Position-Orientation Spacecited by 41
- 2020Dense Steerable Filter CNNs for Exploiting Rotational Symmetry in Histology Imagescited by 33
- 2022VN-Transformer: Rotation-Equivariant Attention for Vector Neuronscited by 27
- 2023E(2)-Equivariant Vision Transformercited by 27
- 2023Truly Scale-Equivariant Deep Nets with Fourier Layerscited by 16
- 2023Euclidean, Projective, Conformal: Choosing a Geometric Algebra for Equivariant Transformerscited by 13
- 2019Co-Attentive Equivariant Neural Networks: Focusing Equivariance On Transformations Co-Occurring In Datacited by 10
- 2017Graph-based Isometry Invariant Representation Learningcited by 7
- 2021Equivariance-bridged SO(2)-Invariant Representation Learning using Graph Convolutional Networkcited by 2
- 2023H-NeXt: The next step towards roto-translation invariant networkscited by 2
Built on by
Citing papers per year: 2025: 1.
1 paper.