mlsanityData to 14 Oct 2026

Equivariance Discovery by Learned Parameter-Sharing

arXiv 2204.03640 ↗First posted 7 Apr 2022cs.LG cs.CVcited by 18

Not on the map. Why?

Abstract (arXiv)

Designing equivariance as an inductive bias into deep-nets has been a prominent approach to build effective models, e.g., a convolutional neural network incorporates translation equivariance. However, incorporating these inductive biases requires knowledge about the equivariance properties of the data, which may not be available, e.g., when encountering a new domain. To address this, we study how to discover interpretable equivariances from data. Specifically, we formulate this discovery process as an optimization problem over a model's parameter-sharing schemes. We propose to use the partition distance to empirically quantify the accuracy of the recovered equivariance. Also, we theoretically analyze the method for Gaussian data and provide a bound on the mean squared gap between the studied discovery scheme and the oracle scheme. Empirically, we show that the approach recovers known equivariances, such as permutations and shifts, on sum of numbers and spatially-invariant data.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 29 of its 47 references that are in the map. First posted on arXiv 7 Apr 2022.

Builds on

29 of 47 references.

References · 29

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  3. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  4. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,089
  5. 2017Deep Setscited by 1,421
  6. 2016Group Equivariant Convolutional Networkscited by 1,165
  7. 2016Geometric deep learning: going beyond Euclidean datacited by 1,048
  8. 2020SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networkscited by 607
  9. 2018Spherical CNNscited by 456
  10. 2015Gradient-based Hyperparameter Optimization through Reversible Learningcited by 360
  11. 2018Invariant and Equivariant Graph Networkscited by 346
  12. 2019Optimizing Millions of Hyperparameters by Implicit Differentiationcited by 304
17 more
  1. 2018Truncated Back-propagation for Bilevel Optimizationcited by 190
  2. 2018Learning Latent Permutations with Gumbel-Sinkhorn Networkscited by 175
  3. 2018Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Networkcited by 171
  4. 2017Equivariance Through Parameter-Sharingcited by 141
  5. 2020On Learning Sets of Symmetric Elementscited by 98
  6. 2020Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphscited by 92
  7. 2020Meta-Learning Symmetries by Reparameterizationcited by 76
  8. 2015Scalable Gradient-Based Tuning of Continuous Regularization Hyperparameterscited by 71
  9. 2018Deep Models of Interactions Across Setscited by 69
  10. 2016TI-POOLING: transformation-invariant pooling for feature learning in Convolutional Neural Networkscited by 67
  11. 2020Group Equivariant Stand-Alone Self-Attention For Visioncited by 63
  12. 2020Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learningcited by 53
  13. 2020Learning Invariances in Neural Networkscited by 52
  14. 2019PIC: Permutation Invariant Critic for Multi-Agent Deep Reinforcement Learningcited by 47
  15. 2019Equivariant Transformer Networkscited by 35
  16. 2019Chirality Nets for Human Pose Regressioncited by 31
  17. 2021Semantic Tracklets: An Object-Centric Representation for Visual Multi-Agent Reinforcement Learningcited by 18

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 7; 2024: 3; 2025: 4; 2026: 2.

18 papers.

Sort:

  1. 2023Learning Layer-wise Equivariances Automatically using Gradientscited by 23
  2. 2022Learnable Polyphase Sampling for Shift Invariant and Equivariant Convolutional Networkscited by 18
  3. 2023IMMA: Immunizing text-to-image Models against Malicious Adaptationcited by 18
  4. 2023Making Vision Transformers Truly Shift-Equivariantcited by 16
  5. 2023Truly Scale-Equivariant Deep Nets with Fourier Layerscited by 16
  6. 2024Relaxed Equivariance via Multitask Learningcited by 13
  7. 2023Alias-Free Convnets: Fractional Shift Invariance via Polynomial Activationscited by 11
  8. 2024On the Ability of Deep Networks to Learn Symmetries from Data: A Neural Kernel Theorycited by 10
  9. 2022Equivariance-aware Architectural Optimization of Neural Networkscited by 8
  10. 2024Learning Symmetries via Weight-Sharing with Doubly Stochastic Tensorscited by 5
  11. 2023PEFT-Ref: A Modular Reference Architecture and Typology for Parameter-Efficient Finetuning Techniquescited by 4
  12. 2025Group Downsampling with Equivariant Anti-aliasingcited by 3
6 more
  1. 2023Weight-Sharing Regularizationcited by 2
  2. 2025Local Scale Equivariance with Latent Deep Equilibrium Canonicalizercited by 2
  3. 2025CLIPSym: Delving into Symmetry Detection with CLIPcited by 2
  4. 2025RECON: Robust symmetry discovery via Explicit Canonical Orientation Normalizationcited by 1
  5. 2026Sparse Data Augmentation for Optimization with Provable Guaranteescited by 1
  6. 2026Tunable Soft Equivariance with Guaranteescited by 0