# Scalars are universal: Equivariant machine learning, structured like classical physics

arXiv: https://arxiv.org/abs/2106.06610 · HTML: https://mlsanity.com/paper/2106.06610/

## Key facts

- Cited by 127 papers in the map (data through 14 Oct 2026).
- Builds on 63 of its 107 references that are in the map.
- First posted on arXiv 11 Jun 2021.

## Builds on (most cited first, 30 of 63)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [E(n) Equivariant Graph Neural Networks](https://mlsanity.com/paper/2102.09844/) (2021)
- [Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks](https://mlsanity.com/paper/1810.02244/) (2018)
- [Convolutional Networks on Graphs for Learning Molecular Fingerprints](https://mlsanity.com/paper/1509.09292/) (2015)
- [Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds](https://mlsanity.com/paper/1802.08219/) (2018)
- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizon](https://mlsanity.com/paper/1811.06128/) (2018)
- [SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks](https://mlsanity.com/paper/2006.10503/) (2020)
- [Deep Learning for 3D Point Clouds: A Survey](https://mlsanity.com/paper/1912.12033/) (2019)
- [Equivariant message passing for the prediction of tensorial properties and molecular spectra](https://mlsanity.com/paper/2102.03150/) (2021)
- [Lagrangian Neural Networks](https://mlsanity.com/paper/2003.04630/) (2020)
- [Invariant and Equivariant Graph Networks](https://mlsanity.com/paper/1812.09902/) (2018)
- [GemNet: Universal Directional Graph Neural Networks for Molecules](https://mlsanity.com/paper/2106.08903/) (2021)
- [Improving Graph Neural Network Expressivity via Subgraph Isomorphism Counting](https://mlsanity.com/paper/2006.09252/) (2020)
- [An Efficient Graph Convolutional Network Technique for the Travelling Salesman Problem](https://mlsanity.com/paper/1906.01227/) (2019)
- [Combinatorial optimization and reasoning with graph neural networks](https://mlsanity.com/paper/2102.09544/) (2021)
- [Gauge Equivariant Convolutional Networks and the Icosahedral CNN](https://mlsanity.com/paper/1902.04615/) (2019)
- [Cormorant: Covariant Molecular Neural Networks](https://mlsanity.com/paper/1906.04015/) (2019)
- [Can Graph Neural Networks Count Substructures?](https://mlsanity.com/paper/2002.04025/) (2020)
- [Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning](https://mlsanity.com/paper/1907.04490/) (2019)
- [Towards Physics-informed Deep Learning for Turbulent Flow Prediction](https://mlsanity.com/paper/1911.08655/) (2019)
- [What graph neural networks cannot learn: depth vs width](https://mlsanity.com/paper/1907.03199/) (2019)

## Built on by (most cited first, 30 of 127)

- [Sign and Basis Invariant Networks for Spectral Graph Representation Learning](https://mlsanity.com/paper/2202.13013/) (2022)
- [e3nn: Euclidean Neural Networks](https://mlsanity.com/paper/2207.09453/) (2022)
- [Equivariance with Learned Canonicalization Functions](https://mlsanity.com/paper/2211.06489/) (2022)
- [On the Expressive Power of Geometric Graph Neural Networks](https://mlsanity.com/paper/2301.09308/) (2023)
- [Theory for Equivariant Quantum Neural Networks](https://mlsanity.com/paper/2210.08566/) (2022)
- [Geometrically Equivariant Graph Neural Networks: A Survey](https://mlsanity.com/paper/2202.07230/) (2022)
- [Physics-Guided Deep Learning for Dynamical Systems: A Survey](https://mlsanity.com/paper/2107.01272/) (2021)
- [Fast, Expressive SE(n) Equivariant Networks through Weight-Sharing in Position-Orientation Space](https://mlsanity.com/paper/2310.02970/) (2023)
- [Learning Physical Dynamics with Subequivariant Graph Neural Networks](https://mlsanity.com/paper/2210.06876/) (2022)
- [Low Dimensional Invariant Embeddings for Universal Geometric Learning](https://mlsanity.com/paper/2205.02956/) (2022)
- [Lorentz-Equivariant Geometric Algebra Transformers for High-Energy Physics](https://mlsanity.com/paper/2405.14806/) (2024)
- [EDGI: Equivariant Diffusion for Planning with Embodied Agents](https://mlsanity.com/paper/2303.12410/) (2023)
- [Direct Molecular Conformation Generation](https://mlsanity.com/paper/2202.01356/) (2022)
- [Guaranteed Conservation of Momentum for Learning Particle-based Fluid Dynamics](https://mlsanity.com/paper/2210.06036/) (2022)
- [The Exact Sample Complexity Gain from Invariances for Kernel Regression](https://mlsanity.com/paper/2303.14269/) (2023)
- [A Survey of Geometric Graph Neural Networks: Data Structures, Models and Applications](https://mlsanity.com/paper/2403.00485/) (2024)
- [A Lorentz-Equivariant Transformer for All of the LHC](https://mlsanity.com/paper/2411.00446/) (2024)
- [PELICAN: Permutation Equivariant and Lorentz Invariant or Covariant Aggregator Network for Particle Physics](https://mlsanity.com/paper/2211.00454/) (2022)
- [Clifford-Steerable Convolutional Neural Networks](https://mlsanity.com/paper/2402.14730/) (2024)
- [Equivariant Networks for Crystal Structures](https://mlsanity.com/paper/2211.15420/) (2022)
- [Explainable Equivariant Neural Networks for Particle Physics: PELICAN](https://mlsanity.com/paper/2307.16506/) (2023)
- [The dark side of the forces: assessing non-conservative force models for atomistic machine learning](https://mlsanity.com/paper/2412.11569/) (2024)
- [Improving Equivariant Model Training via Constraint Relaxation](https://mlsanity.com/paper/2408.13242/) (2024)
- [Is Distance Matrix Enough for Geometric Deep Learning?](https://mlsanity.com/paper/2302.05743/) (2023)
- [Symmetry-Informed Geometric Representation for Molecules, Proteins, and Crystalline Materials](https://mlsanity.com/paper/2306.09375/) (2023)
- [Learning on LoRAs: GL-Equivariant Processing of Low-Rank Weight Spaces for Large Finetuned Models](https://mlsanity.com/paper/2410.04207/) (2024)
- [Graph Neural Networks for Molecules](https://mlsanity.com/paper/2209.05582/) (2022)
- [Equivariance via Minimal Frame Averaging for More Symmetries and Efficiency](https://mlsanity.com/paper/2406.07598/) (2024)
- [Group-invariant max filtering](https://mlsanity.com/paper/2205.14039/) (2022)
- [Dimensionless machine learning: Imposing exact units equivariance](https://mlsanity.com/paper/2204.00887/) (2022)

## Abstract (arXiv)

There has been enormous progress in the last few years in designing neural networks that respect the fundamental symmetries and coordinate freedoms of physical law. Some of these frameworks make use of irreducible representations, some make use of high-order tensor objects, and some apply symmetry-enforcing constraints. Different physical laws obey different combinations of fundamental symmetries, but a large fraction (possibly all) of classical physics is equivariant to translation, rotation, reflection (parity), boost (relativity), and permutations. Here we show that it is simple to parameterize universally approximating polynomial functions that are equivariant under these symmetries, or under the Euclidean, Lorentz, and Poincar'e groups, at any dimensionality d. The key observation is that nonlinear O(d)-equivariant (and related-group-equivariant) functions can be universally expressed in terms of a lightweight collection of scalars -- scalar products and scalar contractions of the scalar, vector, and tensor inputs. We complement our theory with numerical examples that show that the scalar-based method is simple, efficient, and scalable.