mlsanityData to 7 Oct 2026

Group Equivariant Stand-Alone Self-Attention For Vision

arXiv 2010.00977 ↗First posted 2 Oct 2020cs.CV stat.MLcited by 63

Not on the map. Why?

Abstract (arXiv)

We provide a general self-attention formulation to impose group equivariance to arbitrary symmetry groups. This is achieved by defining positional encodings that are invariant to the action of the group considered. Since the group acts on the positional encoding directly, group equivariant self-attention networks (GSA-Nets) are steerable by nature. Our experiments on vision benchmarks demonstrate consistent improvements of GSA-Nets over non-equivariant self-attention networks.

Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 14; 2023: 11; 2024: 5; 2025: 10; 2026: 9.

63 papers.

Sort:

By area:

  1. 2021E(n) Equivariant Graph Neural Networkscited by 951
  2. 2021Frame Averaging for Invariant and Equivariant Network Designcited by 166
  3. 2021CKConv: Continuous Kernel Convolution For Sequential Datacited by 117
  4. 2021Independent SE(3)-Equivariant Models for End-to-End Rigid Protein Dockingcited by 81
  5. 2021Is Attention Better Than Matrix Decomposition?cited by 75
  6. 2023GTA: A Geometry-Aware Attention Mechanism for Multi-View Transformerscited by 75
  7. 2021Universal Approximation of Functions on Setscited by 62
  8. 2021Learning Partial Equivariances from Datacited by 44
  9. 2022Max Pooling with Vision Transformers reconciles class and shape in weakly supervised semantic segmentationcited by 41
  10. 2022Unsupervised Learning of Group Invariant and Equivariant Representationscited by 38
  11. 2021DISCO: accurate Discrete Scale Convolutionscited by 35
  12. 2022INO: Invariant Neural Operators for Learning Complex Physical Systems with Momentum Conservationcited by 28
38 more
  1. 2023E(2)-Equivariant Vision Transformercited by 27
  2. 2022SE(3)-Equivariant Attention Networks for Shape Reconstruction in Function Spacecited by 26
  3. 2021Relative Molecule Self-Attention Transformercited by 21
  4. 2022ScoreNet: Learning Non-Uniform Attention and Augmentation for Transformer-Based Histopathological Image Classificationcited by 21
  5. 2022Unified Fourier-based Kernel and Nonlinearity Design for Equivariant Networks on Homogeneous Spacescited by 20
  6. 2021Frame Averaging for Equivariant Shape Space Learningcited by 18
  7. 2022Equivariance Discovery by Learned Parameter-Sharingcited by 18
  8. 2023Modelling Long Range Dependencies in ND: From Task-Specific to a General Purpose CNNcited by 18
  9. 2021Equivariant Networks for Pixelized Spherescited by 16
  10. 2022An Analysis of Attention via the Lens of Exchangeability and Latent Variable Modelscited by 15
  11. 2022Equivariant Mesh Attention Networkscited by 12
  12. 2023Efficient Equivariant Transfer Learning from Pretrained Modelscited by 11
  13. 2023Exploiting Code Symmetries for Learning Program Semanticscited by 9
  14. 2025Platonic Transformers: A Solid Choice For Equivariancecited by 9
  15. 2023Task-Equivariant Graph Few-shot Learningcited by 8
  16. 2024Steerable Transformers for Volumetric Datacited by 8
  17. 2022Improving the Sample-Complexity of Deep Classification Networks with Invariant Integrationcited by 7
  18. 2024SE(3) Equivariant Ray Embeddings for Implicit Multi-View Depth Estimationcited by 6
  19. 2025Diffusion Generative Modeling on Lie Group Representationscited by 6
  20. 2023Steerable Equivariant Representation Learningcited by 5
  21. 2025Geometric and Physical Constraints Synergistically Enhance Neural PDE Surrogatescited by 5
  22. 2025Rotation Equivariant Arbitrary-scale Image Super-Resolutioncited by 5
  23. 2023Self-Supervised Detection of Perfect and Partial Input-Dependent Symmetriescited by 4
  24. 2022FedMCSA: Personalized Federated Learning via Model Components Self-Attentioncited by 3
  25. 2022Holographic-(V)AE: an end-to-end SO(3)-Equivariant (Variational) Autoencoder in Fourier Spacecited by 3
  26. 2024SiT: Symmetry-Invariant Transformers for Generalisation in Reinforcement Learningcited by 3
  27. 2025Group Downsampling with Equivariant Anti-aliasingcited by 3
  28. 2026Vanilla Group Equivariant Vision Transformer: Simple and Effectivecited by 3
  29. 2021Beyond permutation equivariance in graph networkscited by 2
  30. 20216DCNN with roto-translational convolution filters for volumetric data processingcited by 2
  31. 2023Deep Neural Networks with Efficient Guaranteed Invariancescited by 2
  32. 2023Transformers are Universal Predictorscited by 2
  33. 2023Efficient Rotation Invariance in Deep Neural Networks through Artificial Mental Rotationcited by 2
  34. 2024From Learning to Optimize to Learning Optimization Algorithmscited by 2
  35. 2025A unified framework for establishing the universal approximation of transformer-type architecturescited by 2
  36. 2025Alias-Free ViT: Fractional Shift Invariance via Linear Attentioncited by 2
  37. 2025Concept Regions Matter: Benchmarking CLIP with a New Cluster-Importance Approachcited by 2
  38. 2021Symmetry-driven graph neural networkscited by 1

50 most cited of 63 All 63 (JSON)