mlsanityData to 7 Oct 2026

Drawback of Enforcing Equivariance and its Compensation via the Lens of Expressive Power

arXiv 2512.09673 ↗First posted 10 Dec 2025cs.LG cs.AI cs.NEcited by 0

Not on the map. Why?

Abstract (arXiv)

Equivariant neural networks encode the intrinsic symmetry of data as an inductive bias, which has achieved impressive performance in wide domains. However, the understanding to their expressive power remains premature. Focusing on 2-layer ReLU networks, this paper investigates the impact of enforcing equivariance constraints on the expressive power. By examining the boundary hyperplanes and the channel vectors, we constructively demonstrate that enforcing equivariance constraints could undermine the expressive power. Naturally, this drawback can be compensated for by enlarging the model size -- we further prove upper bounds on the required enlargement for compensation. Surprisingly, we show that the enlarged neural architectures have reduced hypothesis space dimensionality, implying even better generalizability.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 42 of its 51 references that are in the map. First posted on arXiv 10 Dec 2025.

Builds on

42 of 51 references.

References · 42

  1. 2017Graph Attention Networkscited by 8,854
  2. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  3. 2017Deep Setscited by 1,394
  4. 2016Group Equivariant Convolutional Networkscited by 1,154
  5. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
  6. 2018PointCNN: Convolution On X-Transformed Pointscited by 764
  7. 2022Equivariant Diffusion for Molecule Generation in 3Dcited by 666
  8. 2020SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networkscited by 602
  9. 2020Directional Message Passing for Molecular Graphscited by 556
  10. 2019General E(2)-Equivariant Steerable CNNscited by 441
  11. 2018Invariant and Equivariant Graph Networkscited by 339
  12. 2016Steerable CNNscited by 298
30 more
  1. 2016Harmonic Networks: Deep Translation and Rotation Equivariancecited by 293
  2. 2018A General Theory of Equivariant CNNs on Homogeneous Spacescited by 223
  3. 2016Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learningcited by 209
  4. 2017Learning Steerable Filters for Rotation Equivariant CNNscited by 193
  5. 2021A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groupscited by 185
  6. 2019On the Universality of Invariant Networkscited by 151
  7. 2018Universal approximations of invariant maps by neural networkscited by 127
  8. 2020Expressive Power of Invariant and Equivariant Graph Neural Networkscited by 100
  9. 2020LieTransformer: Equivariant self-attention for Lie Groupscited by 100
  10. 2021Provably Strict Generalisation Benefit for Equivariant Modelscited by 90
  11. 2021Equivariant Point Network for 3D Point Cloud Analysiscited by 81
  12. 2022SO(2)-Equivariant Reinforcement Learningcited by 79
  13. 2022On the Symmetries of Deep Learning Models and their Internal Representationscited by 63
  14. 2020Universal Equivariant Multilayer Perceptronscited by 49
  15. 2023Approximation-Generalization Trade-offs under (Approximate) Group Equivariancecited by 45
  16. 2020Recent advances in deep learning theorycited by 41
  17. 2022Unsupervised Learning of Group Invariant and Equivariant Representationscited by 38
  18. 2019Improved Generalization Bounds of Group Invariant / Equivariant Deep Networks via Quotient Feature Spacescited by 34
  19. 2023E(2)-Equivariant Vision Transformercited by 27
  20. 2023Lie Point Symmetry and Physics Informed Networkscited by 24
  21. 2024Improving Equivariant Model Training via Constraint Relaxationcited by 22
  22. 2021Implicit Bias of Linear Equivariant Networkscited by 18
  23. 2024E(3)-Equivariant Mesh Neural Networkscited by 9
  24. 2025Platonic Transformers: A Solid Choice For Equivariancecited by 9
  25. 2023Investigating how ReLU-networks encode symmetriescited by 8
  26. 2022Benefits of Permutation-Equivariance in Auction Mechanismscited by 7
  27. 2022A Classification of G-invariant Shallow Neural Networkscited by 5
  28. 2025On Universality of Deep Equivariant Networkscited by 3
  29. 2022Equivariance Regularization for Image Reconstructioncited by 1
  30. 2023Human-imperceptible, Machine-recognizable Imagescited by 1

Built on by

Not cited yet.