# A tradeoff between universality of equivariant models and learnability of symmetries

arXiv: https://arxiv.org/abs/2210.09444 · HTML: https://mlsanity.com/paper/2210.09444/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 73 references that are in the map.
- First posted on arXiv 17 Oct 2022.

## Builds on (most cited first, 30 of 33)

- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [Spherical CNNs](https://mlsanity.com/paper/1801.10130/) (2018)
- [On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups](https://mlsanity.com/paper/1802.03690/) (2018)
- [Equivariant Flows: Exact Likelihood Generative Learning for Symmetric Densities](https://mlsanity.com/paper/2006.02425/) (2020)
- [Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data](https://mlsanity.com/paper/2002.12880/) (2020)
- [A General Theory of Equivariant CNNs on Homogeneous Spaces](https://mlsanity.com/paper/1811.02017/) (2018)
- [A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groups](https://mlsanity.com/paper/2104.09459/) (2021)
- [Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Network](https://mlsanity.com/paper/1806.09231/) (2018)
- [E(n) Equivariant Normalizing Flows](https://mlsanity.com/paper/2105.09016/) (2021)
- [On the Universality of Invariant Networks](https://mlsanity.com/paper/1901.09342/) (2019)
- [Equivariance Through Parameter-Sharing](https://mlsanity.com/paper/1702.08389/) (2017)
- [Universal approximations of invariant maps by neural networks](https://mlsanity.com/paper/1804.10306/) (2018)
- [Scalars are universal: Equivariant machine learning, structured like classical physics](https://mlsanity.com/paper/2106.06610/) (2021)
- [Lorentz Group Equivariant Neural Network for Particle Physics](https://mlsanity.com/paper/2006.04780/) (2020)
- [Deep Scale-spaces: Equivariance Over Scale](https://mlsanity.com/paper/1905.11697/) (2019)
- [Provably Strict Generalisation Benefit for Equivariant Models](https://mlsanity.com/paper/2102.10333/) (2021)
- [Automatic Symmetry Discovery with Lie Algebra Convolutional Network](https://mlsanity.com/paper/2109.07103/) (2021)
- [On the Universality of Rotation Equivariant Point Cloud Networks](https://mlsanity.com/paper/2010.02449/) (2020)
- [Meta-Learning Symmetries by Reparameterization](https://mlsanity.com/paper/2007.02933/) (2020)
- [Learning Invariances using the Marginal Likelihood](https://mlsanity.com/paper/1808.05563/) (2018)
- [Universal Equivariant Multilayer Perceptrons](https://mlsanity.com/paper/2002.02912/) (2020)
- [Equivariant Hamiltonian Flows](https://mlsanity.com/paper/1909.13739/) (2019)
- [Discovering Symmetry Invariants and Conserved Quantities by Interpreting Siamese Neural Networks](https://mlsanity.com/paper/2003.04299/) (2020)
- [Provably Strict Generalisation Benefit for Invariance in Kernel Methods](https://mlsanity.com/paper/2106.02346/) (2021)
- [Last Layer Marginal Likelihood for Invariance Learning](https://mlsanity.com/paper/2106.07512/) (2021)
- [Implicit Bias of Linear Equivariant Networks](https://mlsanity.com/paper/2110.06084/) (2021)
- [Learning Invariant Weights in Neural Networks](https://mlsanity.com/paper/2202.12439/) (2022)
- [Neural Networks for Learning Counterfactual G-Invariances from Single Environments](https://mlsanity.com/paper/2104.10105/) (2021)

## Built on by (most cited first, 1 of 1)

- [Approximately Equivariant Graph Networks](https://mlsanity.com/paper/2308.10436/) (2023)

## Abstract (arXiv)

We prove an impossibility result, which in the context of function learning says the following: under certain conditions, it is impossible to simultaneously learn symmetries and functions equivariant under them using an ansatz consisting of equivariant functions. To formalize this statement, we carefully study notions of approximation for groups and semigroups. We analyze certain families of neural networks for whether they satisfy the conditions of the impossibility result: what we call ``linearly equivariant'' networks, and group-convolutional networks. A lot can be said precisely about linearly equivariant networks, making them theoretically useful. On the practical side, our analysis of group-convolutional neural networks allows us generalize the well-known ``convolution is all you need'' theorem to non-homogeneous spaces. We additionally find an important difference between group convolution and semigroup convolution.