# Approximate Equivariance via Projection-based Regularisation

arXiv: https://arxiv.org/abs/2601.05028 · HTML: https://mlsanity.com/paper/2601.05028/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 57 references that are in the map.
- First posted on arXiv 8 Jan 2026.

## Builds on (most cited first, 30 of 42)

- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [E(n) Equivariant Graph Neural Networks](https://mlsanity.com/paper/2102.09844/) (2021)
- [Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds](https://mlsanity.com/paper/1802.08219/) (2018)
- [MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classification](https://mlsanity.com/paper/2110.14795/) (2021)
- [Equivariant Diffusion for Molecule Generation in 3D](https://mlsanity.com/paper/2203.17003/) (2022)
- [General E(2)-Equivariant Steerable CNNs](https://mlsanity.com/paper/1911.08251/) (2019)
- [3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data](https://mlsanity.com/paper/1807.02547/) (2018)
- [Steerable CNNs](https://mlsanity.com/paper/1612.08498/) (2016)
- [Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs](https://mlsanity.com/paper/2206.11990/) (2022)
- [Geometric and Physical Quantities Improve E(3) Equivariant Message Passing](https://mlsanity.com/paper/2110.02905/) (2021)
- [EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representations](https://mlsanity.com/paper/2306.12059/) (2023)
- [Lorentz Group Equivariant Neural Network for Particle Physics](https://mlsanity.com/paper/2006.04780/) (2020)
- [Scale-Equivariant Steerable Networks](https://mlsanity.com/paper/1910.11093/) (2019)
- [Approximately Equivariant Networks for Imperfectly Symmetric Dynamics](https://mlsanity.com/paper/2201.11969/) (2022)
- [Provably Strict Generalisation Benefit for Equivariant Models](https://mlsanity.com/paper/2102.10333/) (2021)
- [EQ-VAE: Equivariance Regularized Latent Space for Improved Generative Image Modeling](https://mlsanity.com/paper/2502.09509/) (2025)
- [Learning Smooth and Expressive Interatomic Potentials for Physical Property Prediction](https://mlsanity.com/paper/2502.12147/) (2025)
- [Residual Pathway Priors for Soft Equivariance Constraints](https://mlsanity.com/paper/2112.01388/) (2021)
- [Equivariant Q Learning in Spatial Action Spaces](https://mlsanity.com/paper/2110.15443/) (2021)
- [The Lie Derivative for Measuring Learned Equivariance](https://mlsanity.com/paper/2210.02984/) (2022)
- [Does equivariance matter at scale?](https://mlsanity.com/paper/2410.23179/) (2024)
- [Approximation-Generalization Trade-offs under (Approximate) Group Equivariance](https://mlsanity.com/paper/2305.17592/) (2023)
- [Learning Partial Equivariances from Data](https://mlsanity.com/paper/2110.10211/) (2021)
- [Relaxing Equivariance Constraints with Non-stationary Continuous Filters](https://mlsanity.com/paper/2204.07178/) (2022)
- [Equivariant Graph Neural Operator for Modeling 3D Dynamics](https://mlsanity.com/paper/2401.11037/) (2024)
- [Lorentz-Equivariant Geometric Algebra Transformers for High-Energy Physics](https://mlsanity.com/paper/2405.14806/) (2024)
- [A Lorentz-Equivariant Transformer for All of the LHC](https://mlsanity.com/paper/2411.00446/) (2024)
- [Improving Equivariant Model Training via Constraint Relaxation](https://mlsanity.com/paper/2408.13242/) (2024)
- [A Unified Framework to Enforce, Discover, and Promote Symmetry in Machine Learning](https://mlsanity.com/paper/2311.00212/) (2023)
- [Discovering Symmetry Breaking in Physical Systems with Relaxed Group Convolution](https://mlsanity.com/paper/2310.02299/) (2023)

## Built on by (most cited first, 2 of 2)

- [Recurrent Equivariant Constraint Modulation: Learning Per-Layer Symmetry Relaxation from Data](https://mlsanity.com/paper/2602.02853/) (2026)
- [Rotational Equivariance in Machine Learning: A Comprehensive Tutorial](https://mlsanity.com/paper/2608.31045/) (2026)

## Abstract (arXiv)

Equivariance is a powerful inductive bias in neural networks, improving generalisation and physical consistency. Recently, however, non-equivariant models have regained attention, due to their better runtime performance and imperfect symmetries that might arise in real-world applications. This has motivated the development of approximately equivariant models that strike a middle ground between respecting symmetries and fitting the data distribution. Existing approaches in this field usually apply sample-based regularisers which depend on data augmentation at training time, incurring a high sample complexity, in particular for continuous groups such as SO(3). This work instead approaches approximate equivariance via a projection-based regulariser which leverages the orthogonal decomposition of linear layers into equivariant and non-equivariant components. In contrast to existing methods, this penalises non-equivariance at an operator level across the full group orbit, rather than point-wise. We present a mathematical framework for computing the non-equivariance penalty exactly and efficiently in both the spatial and spectral domain. In our experiments, our method consistently outperforms prior approximate equivariance approaches in both model performance and efficiency, achieving substantial runtime gains over sample-based regularisers.