mlsanityData to 14 Oct 2026

Scalars are universal: Equivariant machine learning, structured like classical physics

arXiv 2106.06610 ↗First posted 11 Jun 2021cs.LG math-ph math.MPcited by 127

Not on the map. Why?

Abstract (arXiv)

There has been enormous progress in the last few years in designing neural networks that respect the fundamental symmetries and coordinate freedoms of physical law. Some of these frameworks make use of irreducible representations, some make use of high-order tensor objects, and some apply symmetry-enforcing constraints. Different physical laws obey different combinations of fundamental symmetries, but a large fraction (possibly all) of classical physics is equivariant to translation, rotation, reflection (parity), boost (relativity), and permutations. Here we show that it is simple to parameterize universally approximating polynomial functions that are equivariant under these symmetries, or under the Euclidean, Lorentz, and Poincar'e groups, at any dimensionality d. The key observation is that nonlinear O(d)-equivariant (and related-group-equivariant) functions can be universally expressed in terms of a lightweight collection of scalars -- scalar products and scalar contractions of the scalar, vector, and tensor inputs. We complement our theory with numerical examples that show that the scalar-based method is simple, efficient, and scalable.

Key facts. Cited by 127 papers in the map (data through 14 Oct 2026). Builds on 63 of its 107 references that are in the map. First posted on arXiv 11 Jun 2021.

Builds on

63 of 107 references.

References · 63

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  2. 2017Graph Attention Networkscited by 8,923
  3. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  4. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  5. 2018How Powerful are Graph Neural Networks?cited by 4,393
  6. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
  7. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,089
  8. 2017Deep Setscited by 1,421
  9. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,167
  10. 2021E(n) Equivariant Graph Neural Networkscited by 965
  11. 2018Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networkscited by 881
  12. 2015Convolutional Networks on Graphs for Learning Molecular Fingerprintscited by 754
38 more
  1. 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 746
  2. 2019Hamiltonian Neural Networkscited by 712
  3. 2018Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizoncited by 623
  4. 2020SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networkscited by 607
  5. 2019Deep Learning for 3D Point Clouds: A Surveycited by 558
  6. 2021Equivariant message passing for the prediction of tensorial properties and molecular spectracited by 441
  7. 2020Lagrangian Neural Networkscited by 398
  8. 2018Invariant and Equivariant Graph Networkscited by 346
  9. 2021GemNet: Universal Directional Graph Neural Networks for Moleculescited by 343
  10. 2020Improving Graph Neural Network Expressivity via Subgraph Isomorphism Countingcited by 313
  11. 2019An Efficient Graph Convolutional Network Technique for the Travelling Salesman Problemcited by 269
  12. 2021Combinatorial optimization and reasoning with graph neural networkscited by 268
  13. 2019Gauge Equivariant Convolutional Networks and the Icosahedral CNNcited by 267
  14. 2019Cormorant: Covariant Molecular Neural Networkscited by 263
  15. 2020Can Graph Neural Networks Count Substructures?cited by 236
  16. 2019Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learningcited by 212
  17. 2019Towards Physics-informed Deep Learning for Turbulent Flow Predictioncited by 204
  18. 2019What graph neural networks cannot learn: depth vs widthcited by 201
  19. 2021A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groupscited by 187
  20. 2019Symplectic ODE-Net: Learning Hamiltonian Dynamics with Controlcited by 184
  21. 2019Universal Invariant and Equivariant Graph Neural Networkscited by 183
  22. 2019On the equivalence between graph isomorphism testing and function approximation with GNNscited by 172
  23. 2019On the Universality of Invariant Networkscited by 152
  24. 2019Relational Pooling for Graph Representationscited by 150
  25. 2017Equivariance Through Parameter-Sharingcited by 141
  26. 2018Universal approximations of invariant maps by neural networkscited by 127
  27. 2020Incorporating Symmetry into Deep Dynamics Models for Improved Generalizationcited by 126
  28. 2020Lorentz Group Equivariant Neural Network for Particle Physicscited by 121
  29. 2020Erdos Goes Neural: an Unsupervised Learning Framework for Combinatorial Optimization on Graphscited by 115
  30. 2019Hamiltonian Graph Networks with ODE Integratorscited by 108
  31. 2020On Learning Sets of Symmetric Elementscited by 98
  32. 2021Provably Strict Generalisation Benefit for Equivariant Modelscited by 91
  33. 2021Learning with invariances in random features and kernel modelscited by 89
  34. 2020Sampling using SU(N) gauge equivariant flowscited by 84
  35. 2015Rotation-invariant convolutional neural networks for galaxy morphology predictioncited by 83
  36. 2019Rotation Invariant Convolutions for 3D Point Clouds Deep Learningcited by 80
  37. 2020On the Universality of Rotation Equivariant Point Cloud Networkscited by 80
  38. 2021Coordinate Independent Convolutional Networks -- Isometry and Gauge Equivariant Convolutions on Riemannian Manifoldscited by 73

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 3; 2022: 27; 2023: 25; 2024: 36; 2025: 19; 2026: 17.

127 papers.

Sort:

By area:

  1. 2022Sign and Basis Invariant Networks for Spectral Graph Representation Learningcited by 179
  2. 2022e3nn: Euclidean Neural Networkscited by 174
  3. 2022Equivariance with Learned Canonicalization Functionscited by 109
  4. 2023On the Expressive Power of Geometric Graph Neural Networkscited by 90
  5. 2022Theory for Equivariant Quantum Neural Networkscited by 62
  6. 2022Geometrically Equivariant Graph Neural Networks: A Surveycited by 56
  7. 2021Physics-Guided Deep Learning for Dynamical Systems: A Surveycited by 44
  8. 2023Fast, Expressive SE(n) Equivariant Networks through Weight-Sharing in Position-Orientation Spacecited by 41
  9. 2022Learning Physical Dynamics with Subequivariant Graph Neural Networkscited by 39
  10. 2022Low Dimensional Invariant Embeddings for Universal Geometric Learningcited by 38
  11. 2024Lorentz-Equivariant Geometric Algebra Transformers for High-Energy Physicscited by 38
  12. 2023EDGI: Equivariant Diffusion for Planning with Embodied Agentscited by 36
38 more
  1. 2022Direct Molecular Conformation Generationcited by 30
  2. 2022Guaranteed Conservation of Momentum for Learning Particle-based Fluid Dynamicscited by 30
  3. 2023The Exact Sample Complexity Gain from Invariances for Kernel Regressioncited by 28
  4. 2024A Survey of Geometric Graph Neural Networks: Data Structures, Models and Applicationscited by 28
  5. 2024A Lorentz-Equivariant Transformer for All of the LHCcited by 28
  6. 2022PELICAN: Permutation Equivariant and Lorentz Invariant or Covariant Aggregator Network for Particle Physicscited by 26
  7. 2024Clifford-Steerable Convolutional Neural Networkscited by 25
  8. 2022Equivariant Networks for Crystal Structurescited by 24
  9. 2023Explainable Equivariant Neural Networks for Particle Physics: PELICANcited by 24
  10. 2024The dark side of the forces: assessing non-conservative force models for atomistic machine learningcited by 24
  11. 2024Improving Equivariant Model Training via Constraint Relaxationcited by 23
  12. 2023Is Distance Matrix Enough for Geometric Deep Learning?cited by 22
  13. 2023Symmetry-Informed Geometric Representation for Molecules, Proteins, and Crystalline Materialscited by 22
  14. 2024Learning on LoRAs: GL-Equivariant Processing of Low-Rank Weight Spaces for Large Finetuned Modelscited by 22
  15. 2022Graph Neural Networks for Moleculescited by 21
  16. 2024Equivariance via Minimal Frame Averaging for More Symmetries and Efficiencycited by 19
  17. 2022Group-invariant max filteringcited by 18
  18. 2022Dimensionless machine learning: Imposing exact units equivariancecited by 17
  19. 2022Brauer's Group Equivariant Neural Networkscited by 17
  20. 2023Approximately Equivariant Graph Networkscited by 16
  21. 2021ZZ-Net: A Universal Rotation Equivariant Architecture for 2D Point Cloudscited by 15
  22. 2022Exact conservation laws for neural network integrators of dynamical systemscited by 15
  23. 2023Expressive Sign Equivariant Networks for Spectral Geometric Learningcited by 15
  24. 2024VN-EGNN: E(3)-Equivariant Graph Neural Networks with Virtual Nodes Enhance Protein Binding Site Identificationcited by 15
  25. 2022Graph Neural Network with Local Frame for Molecular Potential Energy Surfacecited by 14
  26. 2022M"obius Convolutions for Spherical CNNscited by 13
  27. 2022Machine learning and invariant theorycited by 13
  28. 2025Probing Equivariance and Symmetry Breaking in Convolutional Networkscited by 12
  29. 2025UniMoMo: Unified Generative Modeling of 3D Molecules for De Novo Binder Designcited by 12
  30. 2025Lorentz Local Canonicalization: How to Make Any Network Lorentz-Equivariantcited by 12
  31. 2022Group invariant machine learning by fundamental domain projectionscited by 11
  32. 2022A Simple and Universal Rotation Equivariant Point-cloud Networkcited by 11
  33. 2023Spatial Attention Kinetic Networks with E(n)-Equivariancecited by 11
  34. 2023Denoise Pretraining on Nonequilibrium Molecules for Accurate and Transferable Neural Potentialscited by 11
  35. 2023Subequivariant Graph Reinforcement Learning in 3D Environmentscited by 11
  36. 2024On the Completeness of Invariant Geometric Deep Learning Modelscited by 11
  37. 2023Towards fully covariant machine learningcited by 10
  38. 2023Complete Neural Networks for Complete Euclidean Graphscited by 10

50 most cited of 127 All 127 (JSON)