A tradeoff between universality of equivariant models and learnability of symmetries
Not on the map. Why?
Abstract (arXiv)
We prove an impossibility result, which in the context of function learning says the following: under certain conditions, it is impossible to simultaneously learn symmetries and functions equivariant under them using an ansatz consisting of equivariant functions. To formalize this statement, we carefully study notions of approximation for groups and semigroups. We analyze certain families of neural networks for whether they satisfy the conditions of the impossibility result: what we call ``linearly equivariant'' networks, and group-convolutional networks. A lot can be said precisely about linearly equivariant networks, making them theoretically useful. On the practical side, our analysis of group-convolutional neural networks allows us generalize the well-known ``convolution is all you need'' theorem to non-homogeneous spaces. We additionally find an important difference between group convolution and semigroup convolution.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 33 of its 73 references that are in the map. First posted on arXiv 17 Oct 2022.
Builds on
33 of 73 references.
References · 33
- 2017Deep Setscited by 1,394
- 2016Group Equivariant Convolutional Networkscited by 1,154
- 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
- 2018Spherical CNNscited by 452
- 2018On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groupscited by 296
- 2020Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Datacited by 248
- 2020Equivariant Flows: Exact Likelihood Generative Learning for Symmetric Densitiescited by 248
- 2018A General Theory of Equivariant CNNs on Homogeneous Spacescited by 223
- 2021A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groupscited by 185
- 2018Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Networkcited by 168
- 2021E(n) Equivariant Normalizing Flowscited by 165
- 2019On the Universality of Invariant Networkscited by 151
21 more
- 2017Equivariance Through Parameter-Sharingcited by 139
- 2018Universal approximations of invariant maps by neural networkscited by 127
- 2021Scalars are universal: Equivariant machine learning, structured like classical physicscited by 127
- 2020Lorentz Group Equivariant Neural Network for Particle Physicscited by 120
- 2019Deep Scale-spaces: Equivariance Over Scalecited by 113
- 2021Provably Strict Generalisation Benefit for Equivariant Modelscited by 90
- 2021Automatic Symmetry Discovery with Lie Algebra Convolutional Networkcited by 82
- 2020On the Universality of Rotation Equivariant Point Cloud Networkscited by 80
- 2020Meta-Learning Symmetries by Reparameterizationcited by 75
- 2018Learning Invariances using the Marginal Likelihoodcited by 51
- 2020Universal Equivariant Multilayer Perceptronscited by 49
- 2019Equivariant Hamiltonian Flowscited by 39
- 2020Discovering Symmetry Invariants and Conserved Quantities by Interpreting Siamese Neural Networkscited by 33
- 2021Provably Strict Generalisation Benefit for Invariance in Kernel Methodscited by 29
- 2021Last Layer Marginal Likelihood for Invariance Learningcited by 21
- 2021Implicit Bias of Linear Equivariant Networkscited by 18
- 2022Learning Invariant Weights in Neural Networkscited by 16
- 2021Neural Networks for Learning Counterfactual G-Invariances from Single Environmentscited by 8
- 2013Unsupervised Learning of Invariant Representations in Hierarchical Architecturescited by 5
- 2020Lie PCA: Density estimation for symmetric manifoldscited by 4
- 2022On Non-Linear operators for Geometric Deep Learningcited by 3
Built on by
Citing papers per year: 2023: 1.
1 paper.
- 2023Approximately Equivariant Graph Networkscited by 16
- 2023Approximately Equivariant Graph Networks21 Aug 2023