Autoequivariant Network Search via Group Decomposition
Not on the map. Why?
Abstract (arXiv)
Recent works show that group equivariance as an inductive bias improves neural network performance for both classification and generation. However, designing group-equivariant neural networks is challenging when the group of interest is large and is unknown. Moreover, inducing equivariance can significantly reduce the number of independent parameters in a network with fixed feature size, affecting its overall performance. We address these problems by proving a new group-theoretic result in the context of equivariant neural networks that shows that a network is equivariant to a large group if and only if it is equivariant to smaller groups from which it is constructed. Using this result, we design a novel fast group equivariant construction algorithm, and a deep Q-learning-based search algorithm in a reduced search space, yielding what we call autoequivariant networks (AENs). AENs find the right balance between equivariance and network size when tested on new benchmark datasets, G-MNIST and G-Fashion-MNIST, obtained via group transformations on MNIST and Fashion-MNIST respectively that we release. Extending these results to group convolutional neural networks, where we optimize between equivariances, augmentations, and network sizes, we find group equivariance to be the most dominating factor in all high-performing GCNNs on several datasets like CIFAR10, SVHN, RotMNIST, ASL, EMNIST, and KMNIST.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 41 of its 63 references that are in the map. First posted on arXiv 10 Apr 2021.
Builds on
41 of 63 references.
References · 41
- 2017Attention Is All You Needcited by 59,932
- 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
- 2015Spatial Transformer Networkscited by 1,423
- 2016Group Equivariant Convolutional Networkscited by 1,154
- 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
- 2017Dynamic Routing Between Capsulescited by 689
- 2020SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networkscited by 602
- 2018Spherical CNNscited by 452
- 2019General E(2)-Equivariant Steerable CNNscited by 441
- 2018Deep Learning for Classical Japanese Literaturecited by 415
- 2019Making Convolutional Networks Shift-Invariant Againcited by 390
- 2019Provably Powerful Graph Networkscited by 385
29 more
- 2018Invariant and Equivariant Graph Networkscited by 339
- 2016Steerable CNNscited by 298
- 2018On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groupscited by 296
- 2016Harmonic Networks: Deep Translation and Rotation Equivariancecited by 293
- 2016Designing Neural Network Architectures using Reinforcement Learningcited by 277
- 2019Gauge Equivariant Convolutional Networks and the Icosahedral CNNcited by 262
- 2020Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Datacited by 248
- 2020Equivariant Flows: Exact Likelihood Generative Learning for Symmetric Densitiescited by 248
- 2017Learning SO(3) Equivariant Representations with Spherical CNNscited by 244
- 2019Universal Invariant and Equivariant Graph Neural Networkscited by 182
- 2017Equivariance Through Parameter-Sharingcited by 139
- 2020MDP Homomorphic Networks: Group Symmetries in Reinforcement Learningcited by 133
- 2020Lorentz Group Equivariant Neural Network for Particle Physicscited by 120
- 2019Deep Scale-spaces: Equivariance Over Scalecited by 113
- 2019Scale-Equivariant Steerable Networkscited by 110
- 2020LieTransformer: Equivariant self-attention for Lie Groupscited by 100
- 2019B-Spline CNNs on Lie Groupscited by 97
- 2020Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphscited by 90
- 2017Polar Transformer Networkscited by 81
- 2020Meta-Learning Symmetries by Reparameterizationcited by 75
- 2018Explorations in Homeomorphic Variational Auto-Encodingcited by 65
- 2020Attentive Group Equivariant Convolutional Networkscited by 62
- 2020Truly shift-invariant convolutional neural networkscited by 53
- 2020Spin-Weighted Spherical CNNscited by 50
- 2018Group Equivariant Capsule Networkscited by 31
- 2020Group Equivariant Generative Adversarial Networkscited by 19
- 2020Isometric Transformation Invariant and Equivariant Graph Convolutional Networkscited by 17
- 2019Pretrained AI Models: Performativity, Mobility, and Changecited by 11
- 2019Co-Attentive Equivariant Neural Networks: Focusing Equivariance On Transformations Co-Occurring In Datacited by 10
Built on by
Citing papers per year: 2022: 1; 2023: 2.
3 papers.