# Equivariance Discovery by Learned Parameter-Sharing

arXiv: https://arxiv.org/abs/2204.03640 · HTML: https://mlsanity.com/paper/2204.03640/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 47 references that are in the map.
- First posted on arXiv 7 Apr 2022.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [Geometric deep learning: going beyond Euclidean data](https://mlsanity.com/paper/1611.08097/) (2016)
- [SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks](https://mlsanity.com/paper/2006.10503/) (2020)
- [Spherical CNNs](https://mlsanity.com/paper/1801.10130/) (2018)
- [Gradient-based Hyperparameter Optimization through Reversible Learning](https://mlsanity.com/paper/1502.03492/) (2015)
- [Invariant and Equivariant Graph Networks](https://mlsanity.com/paper/1812.09902/) (2018)
- [Optimizing Millions of Hyperparameters by Implicit Differentiation](https://mlsanity.com/paper/1911.02590/) (2019)
- [Truncated Back-propagation for Bilevel Optimization](https://mlsanity.com/paper/1810.10667/) (2018)
- [Learning Latent Permutations with Gumbel-Sinkhorn Networks](https://mlsanity.com/paper/1802.08665/) (2018)
- [Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Network](https://mlsanity.com/paper/1806.09231/) (2018)
- [Equivariance Through Parameter-Sharing](https://mlsanity.com/paper/1702.08389/) (2017)
- [On Learning Sets of Symmetric Elements](https://mlsanity.com/paper/2002.08599/) (2020)
- [Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphs](https://mlsanity.com/paper/2003.05425/) (2020)
- [Meta-Learning Symmetries by Reparameterization](https://mlsanity.com/paper/2007.02933/) (2020)
- [Scalable Gradient-Based Tuning of Continuous Regularization Hyperparameters](https://mlsanity.com/paper/1511.06727/) (2015)
- [Deep Models of Interactions Across Sets](https://mlsanity.com/paper/1803.02879/) (2018)
- [TI-POOLING: transformation-invariant pooling for feature learning in Convolutional Neural Networks](https://mlsanity.com/paper/1604.06318/) (2016)
- [Group Equivariant Stand-Alone Self-Attention For Vision](https://mlsanity.com/paper/2010.00977/) (2020)
- [Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learning](https://mlsanity.com/paper/2007.01293/) (2020)
- [Learning Invariances in Neural Networks](https://mlsanity.com/paper/2010.11882/) (2020)
- [PIC: Permutation Invariant Critic for Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/1911.00025/) (2019)
- [Equivariant Transformer Networks](https://mlsanity.com/paper/1901.11399/) (2019)
- [Chirality Nets for Human Pose Regression](https://mlsanity.com/paper/1911.00029/) (2019)
- [Semantic Tracklets: An Object-Centric Representation for Visual Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2108.03319/) (2021)

## Built on by (most cited first, 18 of 18)

- [Learning Layer-wise Equivariances Automatically using Gradients](https://mlsanity.com/paper/2310.06131/) (2023)
- [Learnable Polyphase Sampling for Shift Invariant and Equivariant Convolutional Networks](https://mlsanity.com/paper/2210.08001/) (2022)
- [IMMA: Immunizing text-to-image Models against Malicious Adaptation](https://mlsanity.com/paper/2311.18815/) (2023)
- [Making Vision Transformers Truly Shift-Equivariant](https://mlsanity.com/paper/2305.16316/) (2023)
- [Truly Scale-Equivariant Deep Nets with Fourier Layers](https://mlsanity.com/paper/2311.02922/) (2023)
- [Relaxed Equivariance via Multitask Learning](https://mlsanity.com/paper/2410.17878/) (2024)
- [Alias-Free Convnets: Fractional Shift Invariance via Polynomial Activations](https://mlsanity.com/paper/2303.08085/) (2023)
- [On the Ability of Deep Networks to Learn Symmetries from Data: A Neural Kernel Theory](https://mlsanity.com/paper/2412.11521/) (2024)
- [Equivariance-aware Architectural Optimization of Neural Networks](https://mlsanity.com/paper/2210.05484/) (2022)
- [Learning Symmetries via Weight-Sharing with Doubly Stochastic Tensors](https://mlsanity.com/paper/2412.04594/) (2024)
- [PEFT-Ref: A Modular Reference Architecture and Typology for Parameter-Efficient Finetuning Techniques](https://mlsanity.com/paper/2304.12410/) (2023)
- [Group Downsampling with Equivariant Anti-aliasing](https://mlsanity.com/paper/2504.17258/) (2025)
- [Weight-Sharing Regularization](https://mlsanity.com/paper/2311.03096/) (2023)
- [Local Scale Equivariance with Latent Deep Equilibrium Canonicalizer](https://mlsanity.com/paper/2508.14187/) (2025)
- [CLIPSym: Delving into Symmetry Detection with CLIP](https://mlsanity.com/paper/2508.14197/) (2025)
- [RECON: Robust symmetry discovery via Explicit Canonical Orientation Normalization](https://mlsanity.com/paper/2505.13289/) (2025)
- [Sparse Data Augmentation for Optimization with Provable Guarantees](https://mlsanity.com/paper/2609.08133/) (2026)
- [Tunable Soft Equivariance with Guarantees](https://mlsanity.com/paper/2603.26657/) (2026)

## Abstract (arXiv)

Designing equivariance as an inductive bias into deep-nets has been a prominent approach to build effective models, e.g., a convolutional neural network incorporates translation equivariance. However, incorporating these inductive biases requires knowledge about the equivariance properties of the data, which may not be available, e.g., when encountering a new domain. To address this, we study how to discover interpretable equivariances from data. Specifically, we formulate this discovery process as an optimization problem over a model's parameter-sharing schemes. We propose to use the partition distance to empirically quantify the accuracy of the recovered equivariance. Also, we theoretically analyze the method for Gaussian data and provide a bound on the mean squared gap between the studied discovery scheme and the oracle scheme. Empirically, we show that the approach recovers known equivariances, such as permutations and shifts, on sum of numbers and spatially-invariant data.