mlsanityData to 14 Oct 2026

E(2)-Equivariant Vision Transformer

arXiv 2306.06722 ↗First posted 11 Jun 2023cs.CV cs.AIcited by 27

Not on the map. Why?

Abstract (arXiv)

Vision Transformer (ViT) has achieved remarkable performance in computer vision. However, positional encoding in ViT makes it substantially difficult to learn the intrinsic equivariance in data. Initial attempts have been made on designing equivariant ViT but are proved defective in some cases in this paper. To address this issue, we design a Group Equivariant Vision Transformer (GE-ViT) via a novel, effective positional encoding operator. We prove that GE-ViT meets all the theoretical requirements of an equivariant neural network. Comprehensive experiments are conducted on standard benchmark datasets, demonstrating that GE-ViT significantly outperforms non-equivariant self-attention networks. The code is available at https://github.com/ZJUCDSYangKaifan/GEVit.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Builds on 43 of its 56 references that are in the map. First posted on arXiv 11 Jun 2023.

Builds on

43 of 56 references.

References · 43

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2020End-to-End Object Detection with Transformerscited by 7,414
  6. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  7. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
  8. 2020Big Bird: Transformers for Longer Sequencescited by 1,684
  9. 2020Reformer: The Efficient Transformercited by 1,668
  10. 2020Linformer: Self-Attention with Linear Complexitycited by 1,474
  11. 2017Deep Setscited by 1,421
  12. 2016Group Equivariant Convolutional Networkscited by 1,165
31 more
  1. 2018Self-Attention with Relative Position Representationscited by 1,120
  2. 2018PointCNN: Convolution On X-Transformed Pointscited by 764
  3. 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 746
  4. 2020SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networkscited by 607
  5. 2018Spherical CNNscited by 456
  6. 2019General E(2)-Equivariant Steerable CNNscited by 446
  7. 2018Rotation Equivariant CNNs for Digital Pathologycited by 422
  8. 2021Conditional Positional Encodings for Vision Transformerscited by 406
  9. 2014Recurrent Models of Visual Attentioncited by 349
  10. 2020Exploring Self-attention for Image Recognitioncited by 313
  11. 2016Steerable CNNscited by 305
  12. 2019On the Relationship between Self-Attention and Convolutional Layerscited by 300
  13. 2016Harmonic Networks: Deep Translation and Rotation Equivariancecited by 295
  14. 2017Learning SO(3) Equivariant Representations with Spherical CNNscited by 248
  15. 2020Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Datacited by 248
  16. 2018A General Theory of Equivariant CNNs on Homogeneous Spacescited by 228
  17. 2020Understanding the Difficulty of Training Transformerscited by 210
  18. 2017Learning Steerable Filters for Rotation Equivariant CNNscited by 194
  19. 2020LieTransformer: Equivariant self-attention for Lie Groupscited by 100
  20. 2016Rotation equivariant vector field networkscited by 99
  21. 2019B-Spline CNNs on Lie Groupscited by 97
  22. 2017Oriented Response Networkscited by 68
  23. 2020Attentive Group Equivariant Convolutional Networkscited by 63
  24. 2020Group Equivariant Stand-Alone Self-Attention For Visioncited by 63
  25. 2020Universal Equivariant Multilayer Perceptronscited by 49
  26. 2020Recent advances in deep learning theorycited by 41
  27. 2019Improved Generalization Bounds of Group Invariant / Equivariant Deep Networks via Quotient Feature Spacescited by 34
  28. 2017Estimating Cosmological Parameters from the Dark Matter Distributioncited by 11
  29. 2019Co-Attentive Equivariant Neural Networks: Focusing Equivariance On Transformations Co-Occurring In Datacited by 10
  30. 2022Benefits of Permutation-Equivariance in Auction Mechanismscited by 8
  31. 2019Building Deep, Equivariant Capsule Networkscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 7; 2025: 8; 2026: 10.

27 papers.

Sort:

By area:

  1. 2024Rumor Detection with a novel graph neural network approachcited by 15
  2. 2024Image Captioning in news report scenariocited by 12
  3. 2023Optimizing PatchCore for Few/many-shot Anomaly Detectioncited by 10
  4. 2024Steerable Transformers for Volumetric Datacited by 8
  5. 2025Flopping for FLOPs: Leveraging equivariance for computational efficiencycited by 8
  6. 2024Learning to Play Atari in a World of Tokenscited by 6
  7. 2025Enhancing Image Restoration Transformer via Adaptive Translation Equivariancecited by 5
  8. 2025Rotation Equivariant Arbitrary-scale Image Super-Resolutioncited by 5
  9. 2025Group Downsampling with Equivariant Anti-aliasingcited by 3
  10. 2025Quick ViTs: Speeding up Vision Transformers through Equivariancecited by 3
  11. 2026Vanilla Group Equivariant Vision Transformer: Simple and Effectivecited by 3
  12. 2023Efficient Rotation Invariance in Deep Neural Networks through Artificial Mental Rotationcited by 2
15 more
  1. 2025CLIPSym: Delving into Symmetry Detection with CLIPcited by 2
  2. 2025Alias-Free ViT: Fractional Shift Invariance via Linear Attentioncited by 2
  3. 2024Contrastive Learning Via Equivariant Representationcited by 1
  4. 2024Harmformer: Harmonic Networks Meet Transformers for Continuous Roto-Translation Equivariancecited by 1
  5. 2026RA-Det: Towards Universal Detection of AI-Generated Images via Robustness Asymmetrycited by 1
  6. 2026Rotation Equivariant Mamba for Vision Taskscited by 1
  7. 2024Adapting Pretrained ViTs with Convolution Injector for Visuo-Motor Controlcited by 0
  8. 2025Drawback of Enforcing Equivariance and its Compensation via the Lens of Expressive Powercited by 0
  9. 2026Equi-ViT: Rotational Equivariant Vision Transformer for Robust Histopathology Analysiscited by 0
  10. 2026ViT Registers and Fractal ViTcited by 0
  11. 2026A Unified Framework for Vision Transformers Equivariant to Discrete Subgroups of O(2)cited by 0
  12. 2026Mirror-Fusion Attention for Reflection-Aware Self-Supervised Representation Learningcited by 0
  13. 2026Flash EQ-Linear: Accelerating Equivariant Linear Layers via Group-wise Discrete Fourier Transformcited by 0
  14. 2026Embedding Rotation Invariance for Provable Multi-Oriented Scene Text Recognitioncited by 0
  15. 2026Inherent Trade-offs when Enforcing Correct Data Symmetries in Multi-head Self-attentioncited by 0