# Geometric Deep Learning and Equivariant Neural Networks

arXiv: https://arxiv.org/abs/2105.13926 · HTML: https://mlsanity.com/paper/2105.13926/

## Key facts

- Cited by 65 papers in the map (data through 14 Oct 2026).
- Builds on 57 of its 84 references that are in the map.
- First posted on arXiv 28 May 2021.

## Builds on (most cited first, 30 of 57)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [Geometric deep learning: going beyond Euclidean data](https://mlsanity.com/paper/1611.08097/) (2016)
- [Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds](https://mlsanity.com/paper/1802.08219/) (2018)
- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [Spherical CNNs](https://mlsanity.com/paper/1801.10130/) (2018)
- [General E(2)-Equivariant Steerable CNNs](https://mlsanity.com/paper/1911.08251/) (2019)
- [Lagrangian Neural Networks](https://mlsanity.com/paper/2003.04630/) (2020)
- [3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data](https://mlsanity.com/paper/1807.02547/) (2018)
- [On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups](https://mlsanity.com/paper/1802.03690/) (2018)
- [Harmonic Networks: Deep Translation and Rotation Equivariance](https://mlsanity.com/paper/1612.04642/) (2016)
- [Gauge Equivariant Convolutional Networks and the Icosahedral CNN](https://mlsanity.com/paper/1902.04615/) (2019)
- [Learning SO(3) Equivariant Representations with Spherical CNNs](https://mlsanity.com/paper/1711.06721/) (2017)
- [A General Theory of Equivariant CNNs on Homogeneous Spaces](https://mlsanity.com/paper/1811.02017/) (2018)
- [Learning Steerable Filters for Rotation Equivariant CNNs](https://mlsanity.com/paper/1711.07289/) (2017)
- [A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groups](https://mlsanity.com/paper/2104.09459/) (2021)
- [Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Network](https://mlsanity.com/paper/1806.09231/) (2018)
- [E(n) Equivariant Normalizing Flows](https://mlsanity.com/paper/2105.09016/) (2021)
- [Hamiltonian Generative Networks](https://mlsanity.com/paper/1909.13789/) (2019)
- [The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation](https://mlsanity.com/paper/1804.09849/) (2018)
- [Rotation equivariant vector field networks](https://mlsanity.com/paper/1612.09346/) (2016)
- [Roto-Translation Covariant Convolutional Networks for Medical Image Analysis](https://mlsanity.com/paper/1804.03393/) (2018)
- [Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphs](https://mlsanity.com/paper/2003.05425/) (2020)
- [Exploiting Cyclic Symmetry in Convolutional Neural Networks](https://mlsanity.com/paper/1602.02660/) (2016)
- [Provably Strict Generalisation Benefit for Equivariant Models](https://mlsanity.com/paper/2102.10333/) (2021)
- [Spherical CNNs on Unstructured Grids](https://mlsanity.com/paper/1901.02039/) (2019)
- [DeepSphere: a graph-based spherical CNN](https://mlsanity.com/paper/2012.15000/) (2020)
- [Attentive Group Equivariant Convolutional Networks](https://mlsanity.com/paper/2002.03830/) (2020)

## Built on by (most cited first, 30 of 65)

- [Neural Conservation Laws: A Divergence-Free Perspective](https://mlsanity.com/paper/2210.01741/) (2022)
- [Symmetry in Neural Network Parameter Spaces](https://mlsanity.com/paper/2506.13018/) (2025)
- [The Exact Sample Complexity Gain from Invariances for Kernel Regression](https://mlsanity.com/paper/2303.14269/) (2023)
- [A case for using rotation invariant features in state of the art feature matchers](https://mlsanity.com/paper/2204.10144/) (2022)
- [Equivariance versus Augmentation for Spherical Images](https://mlsanity.com/paper/2202.03990/) (2022)
- [HEAL-SWIN: A Vision Transformer On The Sphere](https://mlsanity.com/paper/2307.07313/) (2023)
- [Emergent Equivariance in Deep Ensembles](https://mlsanity.com/paper/2403.03103/) (2024)
- [ZZ-Net: A Universal Rotation Equivariant Architecture for 2D Point Clouds](https://mlsanity.com/paper/2111.15341/) (2021)
- [DeltaConv: Anisotropic Operators for Geometric Deep Learning on Point Clouds](https://mlsanity.com/paper/2111.08799/) (2021)
- [Flopping for FLOPs: Leveraging equivariance for computational efficiency](https://mlsanity.com/paper/2502.05169/) (2025)
- [Structure-Preserving Learning Improves Geometry Generalization in Neural PDEs](https://mlsanity.com/paper/2602.02788/) (2026)
- [Understanding the Role of Equivariance in Self-supervised Learning](https://mlsanity.com/paper/2411.06508/) (2024)
- [Geometrical aspects of lattice gauge equivariant convolutional neural networks](https://mlsanity.com/paper/2303.11448/) (2023)
- [Generalized Permutants and Graph GENEOs](https://mlsanity.com/paper/2206.14798/) (2022)
- [Unified theory for joint covariance properties under geometric image transformations for spatio-temporal receptive fields according to the generalized Gaussian derivative model for visual receptive fields](https://mlsanity.com/paper/2311.10543/) (2023)
- [Mathematical Foundation of Interpretable Equivariant Surrogate Models](https://mlsanity.com/paper/2503.01942/) (2025)
- [AtlasD: Automatic Local Symmetry Discovery](https://mlsanity.com/paper/2504.10777/) (2025)
- [Revisiting consistency for semi-supervised semantic segmentation](https://mlsanity.com/paper/2106.07075/) (2021)
- [Variational inference of fractional Brownian motion with linear computational complexity](https://mlsanity.com/paper/2203.07961/) (2022)
- [Equivariant Neural Tangent Kernels](https://mlsanity.com/paper/2406.06504/) (2024)
- [A novel approach to graph distinction through GENEOs and permutants](https://mlsanity.com/paper/2406.08045/) (2024)
- [R2Det: Exploring Relaxed Rotation Equivariance in 2D object detection](https://mlsanity.com/paper/2408.11760/) (2024)
- [Lecture Notes on Normalizing Flows for Lattice Quantum Field Theories](https://mlsanity.com/paper/2504.18126/) (2025)
- [Equivariant neural networks and piecewise linear representation theory](https://mlsanity.com/paper/2408.00949/) (2024)
- [Scale generalisation properties of extended scale-covariant and scale-invariant Gaussian derivative networks on image datasets with spatial scaling variations](https://mlsanity.com/paper/2409.11140/) (2024)
- [Adaptive Canonicalization with Application to Invariant Anisotropic Geometric Networks](https://mlsanity.com/paper/2509.24886/) (2025)
- [Disentangling Dynamical Systems: Causal Representation Learning Meets Local Sparse Attention](https://mlsanity.com/paper/2603.14483/) (2026)
- [Long Scale Error Control in Low Light Image and Video Enhancement Using Equivariance](https://mlsanity.com/paper/2206.01334/) (2022)
- [A Feedforward Unitary Equivariant Neural Network](https://mlsanity.com/paper/2208.12146/) (2022)
- [Equivariant Manifold Neural ODEs and Differential Invariants](https://mlsanity.com/paper/2401.14131/) (2024)

## Abstract (arXiv)

We survey the mathematical foundations of geometric deep learning, focusing on group equivariant and gauge equivariant neural networks. We develop gauge equivariant convolutional neural networks on arbitrary manifolds M using principal bundles with structure group K and equivariant maps between sections of associated vector bundles. We also discuss group equivariant neural networks for homogeneous spaces M=G/K, which are instead equivariant with respect to the global symmetry G on M. Group equivariant layers can be interpreted as intertwiners between induced representations of G, and we show their relation to gauge equivariant convolutional layers. We analyze several applications of this formalism, including semantic segmentation and object detection networks. We also discuss the case of spherical networks in great detail, corresponding to the case M=S2=SO(3)/SO(2). Here we emphasize the use of Fourier analysis involving Wigner matrices, spherical harmonics and Clebsch-Gordan coefficients for G=SO(3), illustrating the power of representation theory for deep learning.