# A Canonicalization Perspective on Invariant and Equivariant Learning

arXiv: https://arxiv.org/abs/2405.18378 · HTML: https://mlsanity.com/paper/2405.18378/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 75 references that are in the map.
- First posted on arXiv 28 May 2024.

## Builds on (most cited first, 30 of 47)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Strategies for Pre-training Graph Neural Networks](https://mlsanity.com/paper/1905.12265/) (2019)
- [Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks](https://mlsanity.com/paper/1810.02244/) (2018)
- [Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds](https://mlsanity.com/paper/1802.08219/) (2018)
- [Benchmarking Graph Neural Networks](https://mlsanity.com/paper/2003.00982/) (2020)
- [SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks](https://mlsanity.com/paper/2006.10503/) (2020)
- [Symmetric Cross Entropy for Robust Learning with Noisy Labels](https://mlsanity.com/paper/1908.06112/) (2019)
- [Principal Neighbourhood Aggregation for Graph Nets](https://mlsanity.com/paper/2004.05718/) (2020)
- [Rethinking Graph Transformers with Spectral Attention](https://mlsanity.com/paper/2106.03893/) (2021)
- [Residual Gated Graph ConvNets](https://mlsanity.com/paper/1711.07553/) (2017)
- [Graph Neural Networks with Learnable Structural and Positional Representations](https://mlsanity.com/paper/2110.07875/) (2021)
- [The Surprising Power of Graph Neural Networks with Random Node Initialization](https://mlsanity.com/paper/2010.01179/) (2020)
- [Sign and Basis Invariant Networks for Spectral Graph Representation Learning](https://mlsanity.com/paper/2202.13013/) (2022)
- [Frame Averaging for Invariant and Equivariant Network Design](https://mlsanity.com/paper/2110.03336/) (2021)
- [On the Universality of Invariant Networks](https://mlsanity.com/paper/1901.09342/) (2019)
- [Relational Pooling for Graph Representations](https://mlsanity.com/paper/1903.02541/) (2019)
- [On The Computational Complexity of Self-Attention](https://mlsanity.com/paper/2209.04881/) (2022)
- [Universal approximations of invariant maps by neural networks](https://mlsanity.com/paper/1804.10306/) (2018)
- [Rethinking the Expressive Power of GNNs via Graph Biconnectivity](https://mlsanity.com/paper/2301.09505/) (2023)
- [Reducing SO(3) Convolutions to SO(2) for Efficient Equivariant GNNs](https://mlsanity.com/paper/2302.03655/) (2023)
- [Understanding and Extending Subgraph GNNs by Rethinking Their Symmetries](https://mlsanity.com/paper/2206.11140/) (2022)
- [Equivariance with Learned Canonicalization Functions](https://mlsanity.com/paper/2211.06489/) (2022)
- [Breaking the Limits of Message Passing Graph Neural Networks](https://mlsanity.com/paper/2106.04319/) (2021)

## Built on by (most cited first, 27 of 27)

- [Scale Equivariant Graph Metanetworks](https://mlsanity.com/paper/2406.10685/) (2024)
- [Lie Algebra Canonicalization: Equivariant Neural Operators under arbitrary Lie Groups](https://mlsanity.com/paper/2410.02698/) (2024)
- [Learning on LoRAs: GL-Equivariant Processing of Low-Rank Weight Spaces for Large Finetuned Models](https://mlsanity.com/paper/2410.04207/) (2024)
- [Relaxed Equivariance via Multitask Learning](https://mlsanity.com/paper/2410.17878/) (2024)
- [Achieving Approximate Symmetry Is Exponentially Easier than Exact Symmetry](https://mlsanity.com/paper/2512.11855/) (2025)
- [Learning with Exact Invariances in Polynomial Time](https://mlsanity.com/paper/2502.19758/) (2025)
- [Learning Inter-Atomic Potentials without Explicit Equivariance](https://mlsanity.com/paper/2510.00027/) (2025)
- [Data Augmentation: A Fourier Analysis Perspective](https://mlsanity.com/paper/2606.24418/) (2026)
- [Spectral Graph Neural Networks are Incomplete on Graphs with a Simple Spectrum](https://mlsanity.com/paper/2506.05530/) (2025)
- [Adaptive Canonicalization with Application to Invariant Anisotropic Geometric Networks](https://mlsanity.com/paper/2509.24886/) (2025)
- [Efficient Learning and Symmetry Discovery under Exact Invariances](https://mlsanity.com/paper/2609.07031/) (2026)
- [Local Scale Equivariance with Latent Deep Equilibrium Canonicalizer](https://mlsanity.com/paper/2508.14187/) (2025)
- [Frame-based Equivariant Diffusion Models for 3D Molecular Generation](https://mlsanity.com/paper/2509.19506/) (2025)
- [ADAPT: Lightweight, Long-Range Machine Learning Force Fields Without Graphs](https://mlsanity.com/paper/2509.24115/) (2025)
- [Robust Canonicalization through Bootstrapped Data Re-Alignment](https://mlsanity.com/paper/2510.08178/) (2025)
- [Sparse Data Augmentation for Optimization with Provable Guarantees](https://mlsanity.com/paper/2609.08133/) (2026)
- [Local-Canonicalization Equivariant Graph Neural Networks for Sample-Efficient and Generalizable Swarm Robot Control](https://mlsanity.com/paper/2509.14431/) (2025)
- [PiCo: Active Manifold Canonicalization for Robust Robotic Visual Anomaly Detection](https://mlsanity.com/paper/2603.23122/) (2026)
- [When and How to Canonize: A Generalization Perspective](https://mlsanity.com/paper/2605.11008/) (2026)
- [PRISMat: Policy-Driven, Permutation-Invariant Autoregressive Material Generation](https://mlsanity.com/paper/2605.16612/) (2026)
- [PACE-FNO: Physics-Aligned Canonical Equivariance for Fourier Neural Operators](https://mlsanity.com/paper/2605.18606/) (2026)
- [Weisfeiler-Leman Is Incomplete on Simple Spectrum Graphs, so Canonicalize Them](https://mlsanity.com/paper/2605.23446/) (2026)
- [Zero-Shot Test-Time Canonicalization using Out-of-Distribution Scoring](https://mlsanity.com/paper/2606.24178/) (2026)
- [TAGTorch: A PyTorch Library for Geometry, Topology, and Symmetry-Aware Machine Learning](https://mlsanity.com/paper/2607.28755/) (2026)
- [GraphK: Variable-Size Graph Generation with Efficient Edge Construction](https://mlsanity.com/paper/2608.18777/) (2026)
- [Edge-Level Automorphism in GNNs: A Quantitative Framework and Effective Designs For Link Prediction](https://mlsanity.com/paper/2609.34729/) (2026)
- [Dimension-Free Scaling Laws for Size Generalization: An Exponential Separation](https://mlsanity.com/paper/2610.19279/) (2026)

## Abstract (arXiv)

In many applications, we desire neural networks to exhibit invariance or equivariance to certain groups due to symmetries inherent in the data. Recently, frame-averaging methods emerged to be a unified framework for attaining symmetries efficiently by averaging over input-dependent subsets of the group, i.e., frames. What we currently lack is a principled understanding of the design of frames. In this work, we introduce a canonicalization perspective that provides an essential and complete view of the design of frames. Canonicalization is a classic approach for attaining invariance by mapping inputs to their canonical forms. We show that there exists an inherent connection between frames and canonical forms. Leveraging this connection, we can efficiently compare the complexity of frames as well as determine the optimality of certain frames. Guided by this principle, we design novel frames for eigenvectors that are strictly superior to existing methods -- some are even optimal -- both theoretically and empirically. The reduction to the canonicalization perspective further uncovers equivalences between previous methods. These observations suggest that canonicalization provides a fundamental understanding of existing frame-averaging methods and unifies existing equivariant and invariant learning methods. Code is available at https://github.com/PKU-ML/canonicalization.