mlsanityData to 7 Oct 2026

Flopping for FLOPs: Leveraging equivariance for computational efficiency

arXiv 2502.05169 ↗First posted 7 Feb 2025cs.CV cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

Incorporating geometric invariance into neural networks enhances parameter efficiency but typically increases computational costs. This paper introduces new equivariant neural networks that preserve symmetry while maintaining a comparable number of floating-point operations (FLOPs) per parameter to standard non-equivariant networks. We focus on horizontal mirroring (flopping) invariance, common in many computer vision tasks. The main idea is to parametrize the feature spaces in terms of mirror-symmetric and mirror-antisymmetric features, i.e., irreps of the flopping group. This decomposes the linear layers to be block-diagonal, requiring half the number of FLOPs. Our approach reduces both FLOPs and wall-clock time, providing a practical solution for efficient, scalable symmetry-aware architectures.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 47 of its 66 references that are in the map. First posted on arXiv 7 Feb 2025.

Builds on

47 of 66 references.

References · 47

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  6. 2020Scaling Laws for Neural Language Modelscited by 6,411
  7. 2016Layer Normalizationcited by 5,579
  8. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  9. 2022A ConvNet for the 2020scited by 3,545
  10. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  11. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  12. 2016Identity Mappings in Deep Residual Networkscited by 1,960
35 more
  1. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  2. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  3. 2016Group Equivariant Convolutional Networkscited by 1,154
  4. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
  5. 2021Going deeper with Image Transformerscited by 759
  6. 2019General E(2)-Equivariant Steerable CNNscited by 441
  7. 2022DeiT III: Revenge of the ViTcited by 417
  8. 2021ResMLP: Feedforward networks for image classification with data-efficient trainingcited by 338
  9. 2016Steerable CNNscited by 298
  10. 2014Understanding image representations by measuring their equivariance and equivalencecited by 240
  11. 2015Training Very Deep Networkscited by 172
  12. 2018Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Networkcited by 168
  13. 2016Exploiting Cyclic Symmetry in Convolutional Neural Networkscited by 91
  14. 2018Roto-Translation Covariant Convolutional Networks for Medical Image Analysiscited by 91
  15. 2024Learning and Leveraging World Models in Visual Representation Learningcited by 71
  16. 2021Geometric Deep Learning and Equivariant Neural Networkscited by 64
  17. 2020Attentive Group Equivariant Convolutional Networkscited by 62
  18. 2021Do You Even Need Attention? A Stack of Feed-Forward Layers Does Surprisingly Well on ImageNetcited by 61
  19. 2022The Lie Derivative for Measuring Learned Equivariancecited by 51
  20. 2024Does equivariance matter at scale?cited by 49
  21. 2023Geometric Algebra Transformercited by 46
  22. 2023Equivariant Adaptation of Large Pretrained Modelscited by 40
  23. 2023Fast, Expressive SE(n) Equivariant Networks through Weight-Sharing in Position-Orientation Spacecited by 40
  24. 2023E(2)-Equivariant Vision Transformercited by 27
  25. 2023Structuring Representation Geometry with Rotationally Equivariant Contrastive Learningcited by 24
  26. 2023Learning Rotation-Equivariant Features for Visual Correspondencecited by 23
  27. 2023Harmonics of Learning: Universal Fourier Features Emerge in Invariant Networkscited by 23
  28. 2021Exploiting Redundancy: Separable Group Convolutional Networks on Lie Groupscited by 22
  29. 2024Improving Equivariant Model Training via Constraint Relaxationcited by 22
  30. 2023Steerers: A framework for rotation equivariant keypoint descriptorscited by 21
  31. 2023What Affects Learned Equivariance in Deep Image Recognition Models?cited by 14
  32. 2023On genuine invariance learning without weight-tyingcited by 13
  33. 2023Investigating how ReLU-networks encode symmetriescited by 8
  34. 2024Symmetry-Based Structured Matrices for Efficient Approximately Equivariant Networkscited by 7
  35. 2021Exploiting Learned Symmetries in Group Equivariant Convolutionscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 6.

8 papers.

  1. 2025Platonic Transformers: A Solid Choice For Equivariancecited by 9
  2. 2026LoMa: Local Feature Matching Revisitedcited by 7
  3. 2025Quick ViTs: Speeding up Vision Transformers through Equivariancecited by 3
  4. 2026Vanilla Group Equivariant Vision Transformer: Simple and Effectivecited by 3
  5. 2026Who Handles Orientation? Investigating Invariance in Feature Matchingcited by 1
  6. 2026Tunable Soft Equivariance with Guaranteescited by 0
  7. 2026A Unified Framework for Vision Transformers Equivariant to Discrete Subgroups of O(2)cited by 0
  8. 2026Flash EQ-Linear: Accelerating Equivariant Linear Layers via Group-wise Discrete Fourier Transformcited by 0