mlsanityData to 14 Oct 2026

Ensembles provably learn equivariance through data augmentation

arXiv 2410.01452 ↗First posted 2 Oct 2024cs.LG cs.NA math.NAcited by 8

Not on the map. Why?

Abstract (arXiv)

Recently, it was proved that group equivariance emerges in ensembles of neural networks as the result of full augmentation in the limit of infinitely wide neural networks (neural tangent kernel limit). In this paper, we extend this result significantly. We provide a proof that this emergence does not depend on the neural tangent kernel limit at all. We also consider stochastic settings, and furthermore general architectures. For the latter, we provide a simple sufficient condition on the relation between the architecture and the action of the group for our results to hold. We validate our findings through simple numeric experiments.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 32 of its 42 references that are in the map. First posted on arXiv 2 Oct 2024.

Builds on

32 of 42 references.

References · 32

  1. 2016Layer Normalizationcited by 5,669
  2. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  3. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  4. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  5. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,434
  6. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,167
  7. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 738
  8. 2020SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networkscited by 607
  9. 2019General E(2)-Equivariant Steerable CNNscited by 446
  10. 2015An Introduction to Matrix Concentration Inequalitiescited by 397
  11. 2018Invariant and Equivariant Graph Networkscited by 346
  12. 2018On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groupscited by 300
20 more
  1. 2020Equivariant Flows: Exact Likelihood Generative Learning for Symmetric Densitiescited by 250
  2. 2018A General Theory of Equivariant CNNs on Homogeneous Spacescited by 228
  3. 2021A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groupscited by 187
  4. 2021E(n) Equivariant Normalizing Flowscited by 165
  5. 2021Geometry of the Loss Landscape in Overparameterized Neural Networks: Symmetries and Invariancescited by 127
  6. 2018A Kernel Theory of Modern Data Augmentationcited by 106
  7. 2021Provably Strict Generalisation Benefit for Equivariant Modelscited by 91
  8. 2021Learning with invariances in random features and kernel modelscited by 89
  9. 2020On the Benefits of Invariance in Neural Networkscited by 79
  10. 2020Why Are Convolutional Nets More Sample-Efficient than Fully-Connected Nets?cited by 34
  11. 2022Equivariance versus Augmentation for Spherical Imagescited by 24
  12. 2023Learning Layer-wise Equivariances Automatically using Gradientscited by 23
  13. 2024Improving Equivariant Model Training via Constraint Relaxationcited by 23
  14. 2021Equivariant Manifold Flowscited by 20
  15. 2023Optimization Dynamics of Equivariant and Augmented Neural Networkscited by 18
  16. 2024Emergent Equivariance in Deep Ensemblescited by 18
  17. 2021Training or Architecture? How to Incorporate Invariance in Neural Networkscited by 9
  18. 2023On the Implicit Bias of Linear Equivariant Steerable Networkscited by 7
  19. 2021Invariance, encodings, and generalization: learning identity effects with neural networkscited by 5
  20. 2024Symmetries in Overparametrized Neural Networks: A Mean-Field Viewcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 3; 2026: 4.

8 papers.

  1. 2024On the Ability of Deep Networks to Learn Symmetries from Data: A Neural Kernel Theorycited by 10
  2. 2025How simple can you go? An off-the-shelf transformer approach to molecular dynamicscited by 10
  3. 2025Data Augmentation and Regularization for Learning Group Equivariancecited by 3
  4. 2025How Ensembles of Distilled Policies Improve Generalisation in Reinforcement Learningcited by 1
  5. 2026Scale-invariant Gaussian derivative residual networkscited by 0
  6. 2026Equivariance and Augmentation for Bayesian Neural Networkscited by 0
  7. 2026Boosting Data Augmentation with Stochastic Weight Averagingcited by 0
  8. 2026Gauge-Equivariant Attention for Rotation-Stable 360^circ Scene Understandingcited by 0