mlsanityData to 7 Oct 2026

Provably Strict Generalisation Benefit for Equivariant Models

arXiv 2102.10333 ↗First posted 20 Feb 2021stat.ML cs.LGcited by 90

Not on the map. Why?

Abstract (arXiv)

It is widely believed that engineering a model to be invariant/equivariant improves generalisation. Despite the growing popularity of this approach, a precise characterisation of the generalisation benefit is lacking. By considering the simplest case of linear models, this paper provides the first provably non-zero improvement in generalisation for invariant/equivariant models when the target distribution is invariant/equivariant with respect to a compact group. Moreover, our work reveals an interesting relationship between generalisation, the number of training examples and properties of the group action. Our results rest on an observation of the structure of function spaces under averaging operators which, along with its consequences for feature averaging, may be of independent interest.

Key facts. Cited by 90 papers in the map (data through 7 Oct 2026). Builds on 19 of its 40 references that are in the map. First posted on arXiv 20 Feb 2021.

Builds on

19 of 40 references.

References · 19

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2014Going Deeper with Convolutionscited by 4,231
  4. 2017Deep Setscited by 1,394
  5. 2016Group Equivariant Convolutional Networkscited by 1,154
  6. 2019Surprises in High-Dimensional Ridgeless Least Squares Interpolationcited by 508
  7. 2018Spherical CNNscited by 452
  8. 2017Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networkscited by 304
  9. 2018On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groupscited by 296
  10. 2018A General Theory of Equivariant CNNs on Homogeneous Spacescited by 223
  11. 2019Ab-Initio Solution of the Many-Electron Schr"odinger Equation with Deep Neural Networkscited by 195
  12. 2019On the Universality of Invariant Networkscited by 151
7 more
  1. 2017Equivariance Through Parameter-Sharingcited by 139
  2. 2018Universal approximations of invariant maps by neural networkscited by 127
  3. 2020On the Benefits of Invariance in Neural Networkscited by 78
  4. 2016Generalization Error of Invariant Classifierscited by 42
  5. 20183D G-CNNs for Pulmonary Nodule Detectioncited by 36
  6. 2015Learning with Group Invariant Features: A Kernel Perspectivecited by 13
  7. 2013Unsupervised Learning of Invariant Representations in Hierarchical Architecturescited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 15; 2023: 18; 2024: 15; 2025: 14; 2026: 18.

90 papers.

Sort:

By area:

  1. 2023SE(3) diffusion model with application to protein backbone generationcited by 214
  2. 2021Scalars are universal: Equivariant machine learning, structured like classical physicscited by 127
  3. 2021Weisfeiler and Leman go Machine Learning: The Story so farcited by 121
  4. 2022Equivariant 3D-Conditional Diffusion Models for Molecular Linker Designcited by 121
  5. 2023Equivariant Architectures for Learning in Deep Weight Spacescited by 106
  6. 2022PAC-Bayes Compression Bounds So Tight That They Can Explain Generalizationcited by 75
  7. 2021Residual Pathway Priors for Soft Equivariance Constraintscited by 74
  8. 2021Geometric Deep Learning and Equivariant Neural Networkscited by 64
  9. 2022Theory for Equivariant Quantum Neural Networkscited by 60
  10. 2024Does equivariance matter at scale?cited by 49
  11. 2023Approximation-Generalization Trade-offs under (Approximate) Group Equivariancecited by 45
  12. 2022Representation Theory for Geometric Quantum Machine Learningcited by 40
38 more
  1. 2021Ab-Initio Potential Energy Surfaces by Pairing GNNs with Neural Wave Functionscited by 36
  2. 2021Provably Strict Generalisation Benefit for Invariance in Kernel Methodscited by 29
  3. 2023The Exact Sample Complexity Gain from Invariances for Kernel Regressioncited by 27
  4. 2021Understanding the Generalization Benefit of Model Invariance from a Data Perspectivecited by 22
  5. 2024Improving Equivariant Model Training via Constraint Relaxationcited by 22
  6. 2022A Theory of PAC Learnability under Transformation Invariancescited by 21
  7. 2021Implicit Bias of Linear Equivariant Networkscited by 18
  8. 2023A General Theory of Correct, Incorrect, and Extrinsic Equivariancecited by 18
  9. 2024Emergent Equivariance in Deep Ensemblescited by 18
  10. 2022On the non-universality of deep learning: quantifying the cost of symmetrycited by 17
  11. 2022A PAC-Bayesian Generalization Bound for Equivariant Networkscited by 17
  12. 2023Optimization Dynamics of Equivariant and Augmented Neural Networkscited by 17
  13. 2022Dimensionless machine learning: Imposing exact units equivariancecited by 16
  14. 2023Approximately Equivariant Graph Networkscited by 16
  15. 2025Improving Equivariant Networks with Probabilistic Symmetry Breakingcited by 15
  16. 2021On the Sample Complexity of Learning under Invariance and Geometric Stabilitycited by 13
  17. 2022Machine learning and invariant theorycited by 13
  18. 2025Probing Equivariance and Symmetry Breaking in Convolutional Networkscited by 12
  19. 2023Towards fully covariant machine learningcited by 10
  20. 2023Investigating how ReLU-networks encode symmetriescited by 8
  21. 2024Approximately Equivariant Neural Processescited by 8
  22. 2024Ensembles provably learn equivariance through data augmentationcited by 8
  23. 2025A Tale of Two Symmetries: Exploring the Loss Landscape of Equivariant Modelscited by 8
  24. 2022Benefits of Permutation-Equivariance in Auction Mechanismscited by 7
  25. 2023On the Implicit Bias of Linear Equivariant Steerable Networkscited by 7
  26. 2024How Deep Networks Learn Sparse and Hierarchical Data: the Sparse Random Hierarchy Modelcited by 7
  27. 2025Koopman-Equivariant Gaussian Processescited by 6
  28. 2026Flow Equivariant World Models: Memory for Partially Observed Dynamic Environmentscited by 6
  29. 2025Learning with Exact Invariances in Polynomial Timecited by 5
  30. 2023MELON: NeRF with Unposed Images in SO(3)cited by 4
  31. 2024Approximately Piecewise E(3) Equivariant Point Networkscited by 4
  32. 2024Equivariant Action Sampling for Reinforcement Learning and Planningcited by 4
  33. 2022Image to Icosahedral Projection for SO(3) Object Reasoning from Single-View Imagescited by 3
  34. 2022Why self-attention is Natural for Sequence-to-Sequence Problems? A Perspective from Symmetriescited by 3
  35. 2023Equivariant geometric convolutions for emulation of dynamical systemscited by 3
  36. 2023Non-parametric Hypothesis Tests for Distributional Group Symmetrycited by 3
  37. 2024Structure Preserving Diffusion Modelscited by 3
  38. 2025Data Augmentation and Regularization for Learning Group Equivariancecited by 3

50 most cited of 90 All 90 (JSON)