# Equivariant neural networks and piecewise linear representation theory

arXiv: https://arxiv.org/abs/2408.00949 · HTML: https://mlsanity.com/paper/2408.00949/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 21 references that are in the map.
- First posted on arXiv 1 Aug 2024.

## Builds on (most cited first, 8 of 8)

- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [Geometric Deep Learning and Equivariant Neural Networks](https://mlsanity.com/paper/2105.13926/) (2021)
- [Theoretical Aspects of Group Equivariant Neural Networks](https://mlsanity.com/paper/2004.05154/) (2020)
- [What is an equivariant neural network?](https://mlsanity.com/paper/2205.07362/) (2022)
- [Fast computation of permutation equivariant layers with the partition algebra](https://mlsanity.com/paper/2303.06208/) (2023)

## Built on by (most cited first, 3 of 3)

- [Piecewise Linear Equivariant Maps for Compact Groups](https://mlsanity.com/paper/2608.21645/) (2026)
- [Graded Representation Theory of Equivariant Neural Networks](https://mlsanity.com/paper/2609.25776/) (2026)
- [Transversal Pooling Neural Networks](https://mlsanity.com/paper/2609.36237/) (2026)

## Abstract (arXiv)

Equivariant neural networks are neural networks with symmetry. Motivated by the theory of group representations, we decompose the layers of an equivariant neural network into simple representations. The nonlinear activation functions lead to interesting nonlinear equivariant maps between simple representations. For example, the rectified linear unit (ReLU) gives rise to piecewise linear maps. We show that these considerations lead to a filtration of equivariant neural networks, generalizing Fourier series. This observation might provide a useful tool for interpreting equivariant neural networks.