mlsanityData to 14 Oct 2026

A PAC-Bayesian Generalization Bound for Equivariant Networks

arXiv 2210.13150 ↗First posted 24 Oct 2022cs.LG stat.MLcited by 18

Not on the map. Why?

Abstract (arXiv)

Equivariant networks capture the inductive bias about the symmetry of the learning task by building those symmetries into the model. In this paper, we study how equivariance relates to generalization error utilizing PAC Bayesian analysis for equivariant networks, where the transformation laws of feature spaces are determined by group representations. By using perturbation analysis of equivariant networks in Fourier domain for each layer, we derive norm-based PAC-Bayesian generalization bounds. The bound characterizes the impact of group size, and multiplicity and degree of irreducible representations on the generalization error and thereby provide a guideline for selecting them. In general, the bound indicates that using larger group size in the model improves the generalization error substantiated by extensive numerical experiments.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 39 of its 66 references that are in the map. First posted on arXiv 24 Oct 2022.

Builds on

39 of 66 references.

References · 39

  1. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  2. 2016Group Equivariant Convolutional Networkscited by 1,165
  3. 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 746
  4. 2017Spectrally-normalized margin bounds for neural networkscited by 686
  5. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 533
  6. 2019Fantastic Generalization Measures and Where to Find Themcited by 513
  7. 2019General E(2)-Equivariant Steerable CNNscited by 446
  8. 20183D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Datacited by 354
  9. 2017A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networkscited by 314
  10. 2018Stronger generalization bounds for deep nets via a compression approachcited by 307
  11. 2016Steerable CNNscited by 305
  12. 2018On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groupscited by 300
27 more
  1. 2016Harmonic Networks: Deep Translation and Rotation Equivariancecited by 295
  2. 2017Size-Independent Sample Complexity of Neural Networkscited by 290
  3. 2020Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Datacited by 248
  4. 2018A General Theory of Equivariant CNNs on Homogeneous Spacescited by 228
  5. 2019Uniform convergence may be unable to explain generalization in deep learningcited by 224
  6. 2017Learning Steerable Filters for Rotation Equivariant CNNscited by 194
  7. 2016Robust Large Margin Deep Neural Networkscited by 118
  8. 2019B-Spline CNNs on Lie Groupscited by 97
  9. 2018Roto-Translation Covariant Convolutional Networks for Medical Image Analysiscited by 92
  10. 2016Exploiting Cyclic Symmetry in Convolutional Neural Networkscited by 91
  11. 2021Provably Strict Generalisation Benefit for Equivariant Modelscited by 91
  12. 2017Entropy-SGD optimizes the prior of a PAC-Bayes bound: Generalization properties of Entropy-SGD and data-dependent priorscited by 81
  13. 2020On the Benefits of Invariance in Neural Networkscited by 79
  14. 2020In Search of Robust Measures of Generalizationcited by 79
  15. 2021Coordinate Independent Convolutional Networks -- Isometry and Gauge Equivariant Convolutions on Riemannian Manifoldscited by 73
  16. 2016TI-POOLING: transformation-invariant pooling for feature learning in Convolutional Neural Networkscited by 67
  17. 2020DeepSphere: a graph-based spherical CNNcited by 66
  18. 2019Improved Sample Complexities for Deep Networks and Robust Classification via an All-Layer Margincited by 56
  19. 2021Uniform Convergence of Interpolators: Gaussian Width, Norm Bounds, and Benign Overfittingcited by 54
  20. 2020Universal Equivariant Multilayer Perceptronscited by 49
  21. 2019In Defense of Uniform Convergence: Generalization via derandomization with an application to interpolating predictorscited by 44
  22. 2016Generalization Error of Invariant Classifierscited by 42
  23. 2020Generalization bounds for deep learningcited by 38
  24. 2019Improved Generalization Bounds of Group Invariant / Equivariant Deep Networks via Quotient Feature Spacescited by 34
  25. 2021Understanding the Generalization Benefit of Model Invariance from a Data Perspectivecited by 22
  26. 2022Non-Vacuous Generalisation Bounds for Shallow Neural Networkscited by 22
  27. 2022The Sample Complexity of One-Hidden-Layer Neural Networkscited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 4; 2024: 5; 2025: 4; 2026: 4.

18 papers.

Sort:

  1. 2024Does equivariance matter at scale?cited by 52
  2. 2023Approximation-Generalization Trade-offs under (Approximate) Group Equivariancecited by 46
  3. 2023A General Theory of Correct, Incorrect, and Extrinsic Equivariancecited by 19
  4. 2023Approximately Equivariant Graph Networkscited by 16
  5. 2024On the hardness of learning under symmetriescited by 13
  6. 2024A Characterization Theorem for Equivariant Networks with Point-wise Activationscited by 9
  7. 2025A Tale of Two Symmetries: Exploring the Loss Landscape of Equivariant Modelscited by 8
  8. 2025Learning with Exact Invariances in Polynomial Timecited by 6
  9. 2026Efficient Learning and Symmetry Discovery under Exact Invariancescited by 3
  10. 2023Learning Polynomial Problems with SL(2,ℝ) Equivariancecited by 2
  11. 2026A PAC-Bayesian approach to generalization for quantum modelscited by 2
  12. 2022Invariant Lipschitz Bandits: A Side Observation Approachcited by 1
6 more
  1. 2024Symmetric Linear Bandits with Hidden Symmetrycited by 1
  2. 2025Symmetries in PAC-Bayesian Learningcited by 1
  3. 2026PAC-Bayes Beyond Parameter Space: Behavioral Equivalence, Z-Information, and Exact Complexity Decompositioncited by 1
  4. 2024Stability Analysis of Equivariant Convolutional Representations Through The Lens of Equivariant Multi-layered CKNscited by 0
  5. 2025Generalization Bounds for Equivariant Networks on Markov Datacited by 0
  6. 2026Specific Algorithmic Interpretability of Neural Networks: A Case Study on Texturescited by 0