mlsanityData to 7 Oct 2026

A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groups

arXiv 2104.09459 ↗First posted 19 Apr 2021cs.LG math.DS stat.MLcited by 185

Not on the map. Why?

Abstract (arXiv)

Symmetries and equivariance are fundamental to the generalization of neural networks on domains such as images, graphs, and point clouds. Existing work has primarily focused on a small number of groups, such as the translation, rotation, and permutation groups. In this work we provide a completely general algorithm for solving for the equivariant layers of matrix groups. In addition to recovering solutions from other works as special cases, we construct multilayer perceptrons equivariant to multiple groups that have never been tackled before, including O(1,3), O(5), Sp(n), and the Rubik's cube group. Our approach outperforms non-equivariant baselines, with applications to particle physics and dynamical systems. We release our software library to enable researchers to construct equivariant layers for arbitrary matrix groups.

Key facts. Cited by 185 papers in the map (data through 7 Oct 2026). Builds on 33 of its 49 references that are in the map. First posted on arXiv 19 Apr 2021.

Builds on

33 of 49 references.

References · 33

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2018Neural Ordinary Differential Equationscited by 3,733
  3. 2017Deep Setscited by 1,394
  4. 2016Group Equivariant Convolutional Networkscited by 1,154
  5. 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 735
  6. 2019Hamiltonian Neural Networkscited by 702
  7. 2020SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networkscited by 602
  8. 2018Spherical CNNscited by 452
  9. 2019General E(2)-Equivariant Steerable CNNscited by 441
  10. 20183D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Datacited by 348
  11. 2018Invariant and Equivariant Graph Networkscited by 339
  12. 2016Steerable CNNscited by 298
21 more
  1. 2019Cormorant: Covariant Molecular Neural Networkscited by 262
  2. 2020Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Datacited by 248
  3. 2018A General Theory of Equivariant CNNs on Homogeneous Spacescited by 223
  4. 2019On the Universality of Invariant Networkscited by 151
  5. 2017Equivariance Through Parameter-Sharingcited by 139
  6. 2020MDP Homomorphic Networks: Group Symmetries in Reinforcement Learningcited by 133
  7. 2020Lorentz Group Equivariant Neural Network for Particle Physicscited by 120
  8. 2019Deep Scale-spaces: Equivariance Over Scalecited by 113
  9. 2016Rotation equivariant vector field networkscited by 98
  10. 2019B-Spline CNNs on Lie Groupscited by 97
  11. 2020On Learning Sets of Symmetric Elementscited by 97
  12. 2017Polar Transformer Networkscited by 81
  13. 2020On the Universality of Rotation Equivariant Point Cloud Networkscited by 80
  14. 2020Universal Equivariant Multilayer Perceptronscited by 49
  15. 2020A Wigner-Eckart Theorem for Group Equivariant Convolution Kernelscited by 46
  16. 2014Global Convergence of Stochastic Gradient Descent for Some Non-convex Matrix Problemscited by 25
  17. 2014A Stochastic PCA and SVD Algorithm with an Exponential Convergence Ratecited by 18
  18. 2015Fast and Simple PCA via Convex Optimizationcited by 16
  19. 2020Set2Graph: Learning Graphs From Setscited by 16
  20. 2020Equivariant Maps for Hierarchical Structurescited by 11
  21. 2020Probabilistic Numeric Convolutional Neural Networkscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 19; 2022: 40; 2023: 54; 2024: 27; 2025: 26; 2026: 19.

185 papers.

Sort:

By area:

  1. 2021Geometric and Physical Quantities Improve E(3) Equivariant Message Passingcited by 225
  2. 2022e3nn: Euclidean Neural Networkscited by 171
  3. 2021Scalars are universal: Equivariant machine learning, structured like classical physicscited by 127
  4. 2022Equivariance with Learned Canonicalization Functionscited by 106
  5. 2023Equivariant Architectures for Learning in Deep Weight Spacescited by 106
  6. 2022Approximately Equivariant Networks for Imperfectly Symmetric Dynamicscited by 95
  7. 2021Automatic Symmetry Discovery with Lie Algebra Convolutional Networkcited by 82
  8. 2021Residual Pathway Priors for Soft Equivariance Constraintscited by 74
  9. 2023Permutation Equivariant Neural Functionalscited by 69
  10. 2021Geometric Deep Learning and Equivariant Neural Networkscited by 64
  11. 2023The No Free Lunch Theorem, Kolmogorov Complexity, and the Role of Inductive Biases in Machine Learningcited by 63
  12. 2021Universal Approximation of Functions on Setscited by 62
38 more
  1. 2022Theory for Equivariant Quantum Neural Networkscited by 60
  2. 2023Graph Metanetworks for Processing Diverse Neural Architecturescited by 55
  3. 2023Clifford Group Equivariant Neural Networkscited by 54
  4. 2023Neural Functional Transformerscited by 50
  5. 2023Approximation-Generalization Trade-offs under (Approximate) Group Equivariancecited by 45
  6. 2021Learning Partial Equivariances from Datacited by 44
  7. 2022Relaxing Equivariance Constraints with Non-stationary Continuous Filterscited by 41
  8. 2023Generative Adversarial Symmetry Discoverycited by 40
  9. 2022Learning Physical Dynamics with Subequivariant Graph Neural Networkscited by 38
  10. 2023Geometric Clifford Algebra Networkscited by 37
  11. 2024Lorentz-Equivariant Geometric Algebra Transformers for High-Energy Physicscited by 36
  12. 2022Deconstructing the Inductive Biases of Hamiltonian Neural Networkscited by 31
  13. 2023Latent Space Symmetry Discoverycited by 29
  14. 2022LieGG: Studying Learned Lie Group Generatorscited by 28
  15. 2023Learning Probabilistic Symmetrization for Architecture Agnostic Equivariancecited by 25
  16. 2024Universal Neural Functionalscited by 25
  17. 2022Equivariant Networks for Crystal Structurescited by 24
  18. 2023Symmetry Breaking and Equivariant Neural Networkscited by 24
  19. 2024Clifford-Steerable Convolutional Neural Networkscited by 24
  20. 2023Learning Layer-wise Equivariances Automatically using Gradientscited by 23
  21. 2021Enabling equivariance for arbitrary Lie groupscited by 22
  22. 2022What is an equivariant neural network?cited by 21
  23. 2023Steerers: A framework for rotation equivariant keypoint descriptorscited by 21
  24. 2024The Empirical Impact of Neural Parameter Symmetries, or Lack Thereofcited by 21
  25. 2024Learning on LoRAs: GL-Equivariant Processing of Low-Rank Weight Spaces for Large Finetuned Modelscited by 21
  26. 2022Unified Fourier-based Kernel and Nonlinearity Design for Equivariant Networks on Homogeneous Spacescited by 20
  27. 2024Beyond Euclid: An Illustrated Guide to Modern Machine Learning with Geometric, Topological, and Algebraic Structurescited by 20
  28. 2022Homomorphism Autoencoder -- Learning Group Structured Representations from Observed Transitionscited by 18
  29. 2022E2PN: Efficient SE(3)-Equivariant Point Networkcited by 17
  30. 2022Brauer's Group Equivariant Neural Networkscited by 17
  31. 2023Optimization Dynamics of Equivariant and Augmented Neural Networkscited by 17
  32. 2022Learning Invariant Weights in Neural Networkscited by 16
  33. 2022Dimensionless machine learning: Imposing exact units equivariancecited by 16
  34. 2022Equivariant Networks for Zero-Shot Coordinationcited by 16
  35. 2023A General Framework for Equivariant Neural Networks on Reductive Lie Groupscited by 16
  36. 2023Regularizing Towards Soft Equivariance Under Mixed Symmetriescited by 16
  37. 2021ZZ-Net: A Universal Rotation Equivariant Architecture for 2D Point Cloudscited by 15
  38. 2022Lorentz group equivariant autoencoderscited by 15

50 most cited of 185 All 185 (JSON)