# The general theory of permutation equivarant neural networks and higher order graph variational encoders

arXiv: https://arxiv.org/abs/2004.03990 · HTML: https://mlsanity.com/paper/2004.03990/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Apr 2020.

## Built on by (most cited first, 28 of 28)

- [Permutation Equivariant Neural Functionals](https://mlsanity.com/paper/2302.14040/) (2023)
- [Theory for Equivariant Quantum Neural Networks](https://mlsanity.com/paper/2210.08566/) (2022)
- [Theoretical Guarantees for Permutation-Equivariant Quantum Neural Networks](https://mlsanity.com/paper/2210.09974/) (2022)
- [Neural Functional Transformers](https://mlsanity.com/paper/2305.13546/) (2023)
- [Autobahn: Automorphism-based Graph Neural Nets](https://mlsanity.com/paper/2103.01710/) (2021)
- [Speeding up Learning Quantum States through Group Equivariant Convolutional Quantum Ans"atze](https://mlsanity.com/paper/2112.07611/) (2021)
- [Universal Neural Functionals](https://mlsanity.com/paper/2402.05232/) (2024)
- [Multiresolution Equivariant Graph Variational Autoencoder](https://mlsanity.com/paper/2106.00967/) (2021)
- [Temporal Multiresolution Graph Neural Networks For Epidemic Prediction](https://mlsanity.com/paper/2205.14831/) (2022)
- [Turbocharging Solution Concepts: Solving NEs, CEs and CCEs with Neural Equilibrium Solvers](https://mlsanity.com/paper/2210.09257/) (2022)
- [Connecting Permutation Equivariant Neural Networks and Partition Diagrams](https://mlsanity.com/paper/2212.08648/) (2022)
- [Steerable Transformers for Volumetric Data](https://mlsanity.com/paper/2405.15932/) (2024)
- [Fast computation of permutation equivariant layers with the partition algebra](https://mlsanity.com/paper/2303.06208/) (2023)
- [A Comparison Between Invariant and Equivariant Classical and Quantum Graph Neural Networks](https://mlsanity.com/paper/2311.18672/) (2023)
- [Unifying O(3) Equivariant Neural Networks Design with Tensor-Network Formalism](https://mlsanity.com/paper/2211.07482/) (2022)
- [Graph Counterfactual Explainable AI via Latent Space Traversal](https://mlsanity.com/paper/2501.08850/) (2025)
- [Adaptive Canonicalization with Application to Invariant Anisotropic Geometric Networks](https://mlsanity.com/paper/2509.24886/) (2025)
- [P-Tensors: a General Formalism for Constructing Higher Order Message Passing Networks](https://mlsanity.com/paper/2306.10767/) (2023)
- [Interpreting Equivariant Representations](https://mlsanity.com/paper/2401.12588/) (2024)
- [Current Symmetry Group Equivariant Convolution Frameworks for Representation Learning](https://mlsanity.com/paper/2409.07327/) (2024)
- [Revisiting Multi-Permutation Equivariance through the Lens of Irreducible Representations](https://mlsanity.com/paper/2410.06665/) (2024)
- [Symmetry-aware Neural Architecture for Embodied Visual Navigation](https://mlsanity.com/paper/2112.09515/) (2021)
- [Equivariant neural networks for recovery of Hadamard matrices](https://mlsanity.com/paper/2201.13157/) (2022)
- [EquiFusion: Kinematics-Agnostic Human Motion Prediction via Equivariant Latent Diffusion](https://mlsanity.com/paper/2607.10984/) (2026)
- [Target-aware Variational Auto-encoders for Ligand Generation with Multimodal Protein Representation Learning](https://mlsanity.com/paper/2309.16685/) (2023)
- [Range-aware Positional Encoding via High-order Pretraining: Theory and Practice](https://mlsanity.com/paper/2409.19117/) (2024)
- [Permutation Equivariant Neural Networks for Symmetric Tensors](https://mlsanity.com/paper/2503.11276/) (2025)
- [Eigenvalues of the Hessian in Deep Learning: The Origin of Symmetry and Its Breaking](https://mlsanity.com/paper/2610.09919/) (2026)

## Abstract (arXiv)

Previous work on symmetric group equivariant neural networks generally only considered the case where the group acts by permuting the elements of a single vector. In this paper we derive formulae for general permutation equivariant layers, including the case where the layer acts on matrices by permuting their rows and columns simultaneously. This case arises naturally in graph learning and relation learning applications. As a specific case of higher order permutation equivariant networks, we present a second order graph variational encoder, and show that the latent distribution of equivariant generative models must be exchangeable. We demonstrate the efficacy of this architecture on the tasks of link prediction in citation graphs and molecular graph generation.