# Densely Connected G-invariant Deep Neural Networks with Signed Permutation Representations

arXiv: https://arxiv.org/abs/2303.04614 · HTML: https://mlsanity.com/paper/2303.04614/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 35 references that are in the map.
- First posted on arXiv 8 Mar 2023.

## Builds on (most cited first, 26 of 26)

- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [Rotation Equivariant CNNs for Digital Pathology](https://mlsanity.com/paper/1806.03962/) (2018)
- [Invariant and Equivariant Graph Networks](https://mlsanity.com/paper/1812.09902/) (2018)
- [Steerable CNNs](https://mlsanity.com/paper/1612.08498/) (2016)
- [On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups](https://mlsanity.com/paper/1802.03690/) (2018)
- [Learning SO(3) Equivariant Representations with Spherical CNNs](https://mlsanity.com/paper/1711.06721/) (2017)
- [A General Theory of Equivariant CNNs on Homogeneous Spaces](https://mlsanity.com/paper/1811.02017/) (2018)
- [A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groups](https://mlsanity.com/paper/2104.09459/) (2021)
- [The Shattered Gradients Problem: If resnets are the answer, then what is the question?](https://mlsanity.com/paper/1702.08591/) (2017)
- [On the Universality of Invariant Networks](https://mlsanity.com/paper/1901.09342/) (2019)
- [Equivariance Through Parameter-Sharing](https://mlsanity.com/paper/1702.08389/) (2017)
- [Geometric Deep Learning on Molecular Representations](https://mlsanity.com/paper/2107.12375/) (2021)
- [Understanding and Improving Convolutional Neural Networks via Concatenated Rectified Linear Units](https://mlsanity.com/paper/1603.05201/) (2016)
- [Spherical CNNs on Unstructured Grids](https://mlsanity.com/paper/1901.02039/) (2019)
- [Universal Equivariant Multilayer Perceptrons](https://mlsanity.com/paper/2002.02912/) (2020)
- [Gauge equivariant neural networks for quantum lattice gauge theories](https://mlsanity.com/paper/2012.05232/) (2020)
- [Equivariant Networks for Crystal Structures](https://mlsanity.com/paper/2211.15420/) (2022)
- [Leveraging Equivariant Features for Absolute Pose Regression](https://mlsanity.com/paper/2204.02163/) (2022)
- [Equivariance-aware Architectural Optimization of Neural Networks](https://mlsanity.com/paper/2210.05484/) (2022)
- [A Classification of G-invariant Shallow Neural Networks](https://mlsanity.com/paper/2205.09219/) (2022)
- [A Computationally Efficient Neural Network Invariant to the Action of Symmetry Subgroups](https://mlsanity.com/paper/2002.07528/) (2020)
- [Autoequivariant Network Search via Group Decomposition](https://mlsanity.com/paper/2104.04848/) (2021)
- [A Group-Equivariant Autoencoder for Identifying Spontaneously Broken Symmetries](https://mlsanity.com/paper/2202.06319/) (2022)

## Built on by (most cited first, 2 of 2)

- [Identifiable Equivariant Networks are Layerwise Equivariant](https://mlsanity.com/paper/2601.21645/) (2026)
- [Pretrain on Small Synthetic Data, Scale Large for Free: Symmetry-Aware Foundation Model for Logic Rule Induction](https://mlsanity.com/paper/2608.00383/) (2026)

## Abstract (arXiv)

We introduce and investigate, for finite groups G, G-invariant deep neural network (G-DNN) architectures with ReLU activation that are densely connected-- i.e., include all possible skip connections. In contrast to other G-invariant architectures in the literature, the preactivations of theG-DNNs presented here are able to transform by signed permutation representations (signed perm-reps) of G. Moreover, the individual layers of the G-DNNs are not required to be G-equivariant; instead, the preactivations are constrained to be G-equivariant functions of the network input in a way that couples weights across all layers. The result is a richer family of G-invariant architectures never seen previously. We derive an efficient implementation of G-DNNs after a reparameterization of weights, as well as necessary and sufficient conditions for an architecture to be ``admissible''-- i.e., nondegenerate and inequivalent to smaller architectures. We include code that allows a user to build a G-DNN interactively layer-by-layer, with the final architecture guaranteed to be admissible. We show that there are far more admissible G-DNN architectures than those accessible with the ``concatenated ReLU'' activation function from the literature. Finally, we apply G-DNNs to two example problems -- (1) multiplication in -1, 1 (with theoretical guarantees) and (2) 3D object classification -- % finding that the inclusion of signed perm-reps significantly boosts predictive performance compared to baselines with only ordinary (i.e., unsigned) perm-reps.