# Exploiting Redundancy: Separable Group Convolutional Networks on Lie Groups

arXiv: https://arxiv.org/abs/2110.13059 · HTML: https://mlsanity.com/paper/2110.13059/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 44 references that are in the map.
- First posted on arXiv 25 Oct 2021.

## Builds on (most cited first, 30 of 33)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [Geometric deep learning: going beyond Euclidean data](https://mlsanity.com/paper/1611.08097/) (2016)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [PointConv: Deep Convolutional Networks on 3D Point Clouds](https://mlsanity.com/paper/1811.07246/) (2018)
- [Network In Network](https://mlsanity.com/paper/1312.4400/) (2013)
- [General E(2)-Equivariant Steerable CNNs](https://mlsanity.com/paper/1911.08251/) (2019)
- [3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data](https://mlsanity.com/paper/1807.02547/) (2018)
- [Steerable CNNs](https://mlsanity.com/paper/1612.08498/) (2016)
- [Harmonic Networks: Deep Translation and Rotation Equivariance](https://mlsanity.com/paper/1612.04642/) (2016)
- [Learning SO(3) Equivariant Representations with Spherical CNNs](https://mlsanity.com/paper/1711.06721/) (2017)
- [Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data](https://mlsanity.com/paper/2002.12880/) (2020)
- [Learning Steerable Filters for Rotation Equivariant CNNs](https://mlsanity.com/paper/1711.07289/) (2017)
- [CKConv: Continuous Kernel Convolution For Sequential Data](https://mlsanity.com/paper/2102.02611/) (2021)
- [Deep Scale-spaces: Equivariance Over Scale](https://mlsanity.com/paper/1905.11697/) (2019)
- [Scale-Equivariant Steerable Networks](https://mlsanity.com/paper/1910.11093/) (2019)
- [Rotation equivariant vector field networks](https://mlsanity.com/paper/1612.09346/) (2016)
- [B-Spline CNNs on Lie Groups](https://mlsanity.com/paper/1909.12057/) (2019)
- [Roto-Translation Covariant Convolutional Networks for Medical Image Analysis](https://mlsanity.com/paper/1804.03393/) (2018)
- [CubeNet: Equivariance to 3D Rotation and Translation](https://mlsanity.com/paper/1804.04458/) (2018)
- [Rigid-Motion Scattering for Texture Classification](https://mlsanity.com/paper/1403.1687/) (2014)
- [3D G-CNNs for Pulmonary Nodule Detection](https://mlsanity.com/paper/1804.04656/) (2018)
- [DISCO: accurate Discrete Scale Convolutions](https://mlsanity.com/paper/2106.02733/) (2021)
- [Scale Equivariance Improves Siamese Tracking](https://mlsanity.com/paper/2007.09115/) (2020)
- [Rethinking Depthwise Separable Convolutions: How Intra-Kernel Correlations Lead to Improved MobileNets](https://mlsanity.com/paper/2003.13549/) (2020)

## Built on by (most cited first, 22 of 22)

- [Relaxing Equivariance Constraints with Non-stationary Continuous Filters](https://mlsanity.com/paper/2204.07178/) (2022)
- [Fast, Expressive SE(n) Equivariant Networks through Weight-Sharing in Position-Orientation Space](https://mlsanity.com/paper/2310.02970/) (2023)
- [Learning Layer-wise Equivariances Automatically using Gradients](https://mlsanity.com/paper/2310.06131/) (2023)
- [Discovering Symmetry Breaking in Physical Systems with Relaxed Group Convolution](https://mlsanity.com/paper/2310.02299/) (2023)
- [Modelling Long Range Dependencies in ND: From Task-Specific to a General Purpose CNN](https://mlsanity.com/paper/2301.10540/) (2023)
- [Space-Time Continuous PDE Forecasting using Equivariant Neural Fields](https://mlsanity.com/paper/2406.06660/) (2024)
- [Towards a General Purpose CNN for Long Range Dependencies in ND](https://mlsanity.com/paper/2206.03398/) (2022)
- [Lie Group Decompositions for Equivariant Neural Networks](https://mlsanity.com/paper/2310.11366/) (2023)
- [Probing Equivariance and Symmetry Breaking in Convolutional Networks](https://mlsanity.com/paper/2501.01999/) (2025)
- [Approximate Equivariance in Reinforcement Learning](https://mlsanity.com/paper/2411.04225/) (2024)
- [Flopping for FLOPs: Leveraging equivariance for computational efficiency](https://mlsanity.com/paper/2502.05169/) (2025)
- [Regular SE(3) Group Convolutions for Volumetric Medical Image Analysis](https://mlsanity.com/paper/2306.13960/) (2023)
- [Lie Neurons: Adjoint-Equivariant Neural Networks for Semisimple Lie Algebras](https://mlsanity.com/paper/2310.04521/) (2023)
- [Learning Symmetries via Weight-Sharing with Doubly Stochastic Tensors](https://mlsanity.com/paper/2412.04594/) (2024)
- [Structure Preserving Diffusion Models](https://mlsanity.com/paper/2402.19369/) (2024)
- [ILPO-NET: Network for the invariant recognition of arbitrary volumetric patterns in 3D](https://mlsanity.com/paper/2403.19612/) (2024)
- [An Exploration of Conditioning Methods in Graph Neural Networks](https://mlsanity.com/paper/2305.01933/) (2023)
- [H-NeXt: The next step towards roto-translation invariant networks](https://mlsanity.com/paper/2311.01111/) (2023)
- [Current Symmetry Group Equivariant Convolution Frameworks for Representation Learning](https://mlsanity.com/paper/2409.07327/) (2024)
- [On the Fourier analysis in the SO(3) space : EquiLoPO Network](https://mlsanity.com/paper/2404.15979/) (2024)
- [Affine Invariance in Continuous-Domain Convolutional Neural Networks](https://mlsanity.com/paper/2311.09245/) (2023)
- [Discretizing Group-Convolutional Neural Networks for 3D Geometry in Feature Space](https://mlsanity.com/paper/2605.15368/) (2026)

## Abstract (arXiv)

Group convolutional neural networks (G-CNNs) have been shown to increase parameter efficiency and model accuracy by incorporating geometric inductive biases. In this work, we investigate the properties of representations learned by regular G-CNNs, and show considerable parameter redundancy in group convolution kernels. This finding motivates further weight-tying by sharing convolution kernels over subgroups. To this end, we introduce convolution kernels that are separable over the subgroup and channel dimensions. In order to obtain equivariance to arbitrary affine Lie groups we provide a continuous parameterisation of separable convolution kernels. We evaluate our approach across several vision datasets, and show that our weight sharing leads to improved performance and computational efficiency. In many settings, separable G-CNNs outperform their non-separable counterpart, while only using a fraction of their training time. In addition, thanks to the increase in computational efficiency, we are able to implement G-CNNs equivariant to the Sim(2) group; the group of dilations, rotations and translations. Sim(2)-equivariance further improves performance on all tasks considered.