# Learning Linear Groups in Neural Networks

arXiv: https://arxiv.org/abs/2305.18552 · HTML: https://mlsanity.com/paper/2305.18552/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 23 references that are in the map.
- First posted on arXiv 29 May 2023.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [Steerable CNNs](https://mlsanity.com/paper/1612.08498/) (2016)
- [On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups](https://mlsanity.com/paper/1802.03690/) (2018)
- [Harmonic Networks: Deep Translation and Rotation Equivariance](https://mlsanity.com/paper/1612.04642/) (2016)
- [Rotation equivariant vector field networks](https://mlsanity.com/paper/1612.09346/) (2016)
- [Exploiting Cyclic Symmetry in Convolutional Neural Networks](https://mlsanity.com/paper/1602.02660/) (2016)
- [Automatic Symmetry Discovery with Lie Algebra Convolutional Network](https://mlsanity.com/paper/2109.07103/) (2021)
- [Meta-Learning Symmetries by Reparameterization](https://mlsanity.com/paper/2007.02933/) (2020)
- [Learning Invariances in Neural Networks](https://mlsanity.com/paper/2010.11882/) (2020)
- [Spin-Weighted Spherical CNNs](https://mlsanity.com/paper/2006.10731/) (2020)
- [Learning Partial Equivariances from Data](https://mlsanity.com/paper/2110.10211/) (2021)
- [Learning the Irreducible Representations of Commutative Lie Groups](https://mlsanity.com/paper/1402.4437/) (2014)
- [Implicit Rank-Minimizing Autoencoder](https://mlsanity.com/paper/2010.00679/) (2020)
- [LieGG: Studying Learned Lie Group Generators](https://mlsanity.com/paper/2210.04345/) (2022)
- [On Multi-Layer Basis Pursuit, Efficient Algorithms and Convolutional Neural Networks](https://mlsanity.com/paper/1806.00701/) (2018)
- [Rethinking the CSC Model for Natural Images](https://mlsanity.com/paper/1909.05742/) (2019)
- [Bispectral Neural Networks](https://mlsanity.com/paper/2209.03416/) (2022)
- [Discriminative Recurrent Sparse Auto-Encoders](https://mlsanity.com/paper/1301.3775/) (2013)

## Abstract (arXiv)

Employing equivariance in neural networks leads to greater parameter efficiency and improved generalization performance through the encoding of domain knowledge in the architecture; however, the majority of existing approaches require an a priori specification of the desired symmetries. We present a neural network architecture, Linear Group Networks (LGNs), for learning linear groups acting on the weight space of neural networks. Linear groups are desirable due to their inherent interpretability, as they can be represented as finite matrices. LGNs learn groups without any supervision or knowledge of the hidden symmetries in the data and the groups can be mapped to well known operations in machine learning. We use LGNs to learn groups on multiple datasets while considering different downstream tasks; we demonstrate that the linear group structure depends on both the data distribution and the considered task.