# Improving Equivariant Networks with Probabilistic Symmetry Breaking

arXiv: https://arxiv.org/abs/2503.21985 · HTML: https://mlsanity.com/paper/2503.21985/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 67 references that are in the map.
- First posted on arXiv 27 Mar 2025.

## Builds on (most cited first, 30 of 44)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [MoleculeNet: A Benchmark for Molecular Machine Learning](https://mlsanity.com/paper/1703.00564/) (2017)
- [E(n) Equivariant Graph Neural Networks](https://mlsanity.com/paper/2102.09844/) (2021)
- [Benchmarking Graph Neural Networks](https://mlsanity.com/paper/2003.00982/) (2020)
- [DiGress: Discrete Denoising diffusion for graph generation](https://mlsanity.com/paper/2209.14734/) (2022)
- [General E(2)-Equivariant Steerable CNNs](https://mlsanity.com/paper/1911.08251/) (2019)
- [Rotation Equivariant CNNs for Digital Pathology](https://mlsanity.com/paper/1806.03962/) (2018)
- [Attention Augmented Convolutional Networks](https://mlsanity.com/paper/1904.09925/) (2019)
- [Graph Neural Networks with Learnable Structural and Positional Representations](https://mlsanity.com/paper/2110.07875/) (2021)
- [Molecular Sets (MOSES): A Benchmarking Platform for Molecular Generation Models](https://mlsanity.com/paper/1811.12823/) (2018)
- [Crystal Diffusion Variational Autoencoder for Periodic Material Generation](https://mlsanity.com/paper/2110.06197/) (2021)
- [Position-aware Graph Neural Networks](https://mlsanity.com/paper/1906.04817/) (2019)
- [The Surprising Power of Graph Neural Networks with Random Node Initialization](https://mlsanity.com/paper/2010.01179/) (2020)
- [Fast Differentiable Sorting and Ranking](https://mlsanity.com/paper/2002.08871/) (2020)
- [Random Features Strengthen Graph Neural Networks](https://mlsanity.com/paper/2002.03155/) (2020)
- [Sign and Basis Invariant Networks for Spectral Graph Representation Learning](https://mlsanity.com/paper/2202.13013/) (2022)
- [Frame Averaging for Invariant and Equivariant Network Design](https://mlsanity.com/paper/2110.03336/) (2021)
- [Graph Inductive Biases in Transformers without Message Passing](https://mlsanity.com/paper/2305.17589/) (2023)
- [Labeling Trick: A Theory of Using Graph Neural Networks for Multi-Node Representation Learning](https://mlsanity.com/paper/2010.16103/) (2020)
- [Equivariance with Learned Canonicalization Functions](https://mlsanity.com/paper/2211.06489/) (2022)
- [Graph Normalizing Flows](https://mlsanity.com/paper/1905.13177/) (2019)
- [Provably Strict Generalisation Benefit for Equivariant Models](https://mlsanity.com/paper/2102.10333/) (2021)
- [Everything is Connected: Graph Neural Networks](https://mlsanity.com/paper/2301.08210/) (2023)
- [Equivariant Frames and the Impossibility of Continuous Canonicalization](https://mlsanity.com/paper/2402.16077/) (2024)
- [Equivariant Adaptation of Large Pretrained Models](https://mlsanity.com/paper/2310.01647/) (2023)
- [Top-N: Equivariant set and graph generation without exchangeability](https://mlsanity.com/paper/2110.02096/) (2021)

## Built on by (most cited first, 16 of 16)

- [Platonic Transformers: A Solid Choice For Equivariance](https://mlsanity.com/paper/2510.03511/) (2025)
- [Learning Flexible Forward Trajectories for Masked Molecular Diffusion](https://mlsanity.com/paper/2505.16790/) (2025)
- [Rethinking Diffusion Models with Symmetries through Canonicalization with Applications to Molecular Graph Generation](https://mlsanity.com/paper/2602.15022/) (2026)
- [Equivariant Flow Matching for Symmetry-Breaking Bifurcation Problems](https://mlsanity.com/paper/2509.03340/) (2025)
- [EquiReg: Equivariance Regularized Diffusion for Inverse Problems](https://mlsanity.com/paper/2505.22973/) (2025)
- [Spectral Graph Neural Networks are Incomplete on Graphs with a Simple Spectrum](https://mlsanity.com/paper/2506.05530/) (2025)
- [Adaptive Canonicalization with Application to Invariant Anisotropic Geometric Networks](https://mlsanity.com/paper/2509.24886/) (2025)
- [Inverting Data Transformations via Diffusion Sampling](https://mlsanity.com/paper/2602.08267/) (2026)
- [Any-Subgroup Equivariant Networks via Symmetry Breaking](https://mlsanity.com/paper/2603.19486/) (2026)
- [Bifurcation Models: Learning Set-Valued Solution Maps with Weight-Tied Dynamics](https://mlsanity.com/paper/2605.07277/) (2026)
- [Energy Loss Functions for Physical Systems](https://mlsanity.com/paper/2511.02087/) (2025)
- [Diffeomorphism-Equivariant Neural Networks](https://mlsanity.com/paper/2602.06695/) (2026)
- [Balancing Symmetry and Efficiency in Graph Flow Matching](https://mlsanity.com/paper/2602.18084/) (2026)
- [Terminal Symmetry as a Carrier of Asymmetric Process Knowledge: Statewise Refinement for Anytime Verified Construction](https://mlsanity.com/paper/2608.11318/) (2026)
- [Graph Matching Relaxations and Amortization for Supervised Graph Prediction](https://mlsanity.com/paper/2609.15437/) (2026)
- [Your Discrete Graph Diffusion Model is Secretly Memorizing](https://mlsanity.com/paper/2610.13786/) (2026)

## Abstract (arXiv)

Equivariance encodes known symmetries into neural networks, often enhancing generalization. However, equivariant networks cannot break symmetries: the output of an equivariant network must, by definition, have at least the same self-symmetries as the input. This poses an important problem, both (1) for prediction tasks on domains where self-symmetries are common, and (2) for generative models, which must break symmetries in order to reconstruct from highly symmetric latent spaces. This fundamental limitation can be addressed by considering equivariant conditional distributions, instead of equivariant functions. We present novel theoretical results that establish necessary and sufficient conditions for representing such distributions. Concretely, this representation provides a practical framework for breaking symmetries in any equivariant network via randomized canonicalization. Our method, SymPE (Symmetry-breaking Positional Encodings), admits a simple interpretation in terms of positional encodings. This approach expands the representational power of equivariant networks while retaining the inductive bias of symmetry, which we justify through generalization bounds. Experimental results demonstrate that SymPE significantly improves performance of group-equivariant and graph neural networks across diffusion models for graphs, graph autoencoders, and lattice spin system modeling.