# Improved Generalization Bounds of Group Invariant / Equivariant Deep Networks via Quotient Feature Spaces

arXiv: https://arxiv.org/abs/1910.06552 · HTML: https://mlsanity.com/paper/1910.06552/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Oct 2019.

## Built on by (most cited first, 30 of 34)

- [Theory for Equivariant Quantum Neural Networks](https://mlsanity.com/paper/2210.08566/) (2022)
- [Does equivariance matter at scale?](https://mlsanity.com/paper/2410.23179/) (2024)
- [Approximation-Generalization Trade-offs under (Approximate) Group Equivariance](https://mlsanity.com/paper/2305.17592/) (2023)
- [Out-of-distribution Generalization with Causal Invariant Transformations](https://mlsanity.com/paper/2203.11528/) (2022)
- [Imaging with Equivariant Deep Learning](https://mlsanity.com/paper/2209.01725/) (2022)
- [The Exact Sample Complexity Gain from Invariances for Kernel Regression](https://mlsanity.com/paper/2303.14269/) (2023)
- [E(2)-Equivariant Vision Transformer](https://mlsanity.com/paper/2306.06722/) (2023)
- [A General Theory of Correct, Incorrect, and Extrinsic Equivariance](https://mlsanity.com/paper/2303.04745/) (2023)
- [A PAC-Bayesian Generalization Bound for Equivariant Networks](https://mlsanity.com/paper/2210.13150/) (2022)
- [Emergent Equivariance in Deep Ensembles](https://mlsanity.com/paper/2403.03103/) (2024)
- [Data Augmentation vs. Equivariant Networks: A Theory of Generalization on Dynamics Forecasting](https://mlsanity.com/paper/2206.09450/) (2022)
- [Automatic Data Augmentation via Invariance-Constrained Learning](https://mlsanity.com/paper/2209.15031/) (2022)
- [An Analysis of Attention via the Lens of Exchangeability and Latent Variable Models](https://mlsanity.com/paper/2212.14852/) (2022)
- [Relational Reasoning via Set Transformers: Provable Efficiency and Applications to MARL](https://mlsanity.com/paper/2209.09845/) (2022)
- [FINDE: Neural Differential Equations for Finding and Preserving Invariant Quantities](https://mlsanity.com/paper/2210.00272/) (2022)
- [A Characterization Theorem for Equivariant Networks with Point-wise Activations](https://mlsanity.com/paper/2401.09235/) (2024)
- [Hyperparameter Optimization through Neural Network Partitioning](https://mlsanity.com/paper/2304.14766/) (2023)
- [On the Implicit Bias of Linear Equivariant Steerable Networks](https://mlsanity.com/paper/2303.04198/) (2023)
- [Predicting Out-of-Domain Generalization with Neighborhood Invariance](https://mlsanity.com/paper/2207.02093/) (2022)
- [Noncommutative C^*-algebra Net: Learning Neural Networks with Powerful Product Structure in C^*-algebra](https://mlsanity.com/paper/2302.01191/) (2023)
- [Capacity of Group-invariant Linear Readouts from Equivariant Representations: How Many Objects can be Linearly Classified Under All Possible Views?](https://mlsanity.com/paper/2110.07472/) (2021)
- [Invariant Lipschitz Bandits: A Side Observation Approach](https://mlsanity.com/paper/2212.07524/) (2022)
- [VC dimensions of group convolutional neural networks](https://mlsanity.com/paper/2212.09507/) (2022)
- [On the VC dimension of deep group convolutional neural networks](https://mlsanity.com/paper/2410.15800/) (2024)
- [On Uncertainty Calibration for Equivariant Functions](https://mlsanity.com/paper/2510.21691/) (2025)
- [Quantitative Approximation Rates for Group Equivariant Learning](https://mlsanity.com/paper/2602.20370/) (2026)
- [Approximating invariant functions with the sorting trick is theoretically justified](https://mlsanity.com/paper/2403.01671/) (2024)
- [Decomposition of Equivariant Maps via Invariant Maps: Application to Universal Approximation under Symmetry](https://mlsanity.com/paper/2409.16922/) (2024)
- [Equivariant symmetry-aware head pose estimation for fetal MRI](https://mlsanity.com/paper/2512.04890/) (2025)
- [Drawback of Enforcing Equivariance and its Compensation via the Lens of Expressive Power](https://mlsanity.com/paper/2512.09673/) (2025)

## Abstract (arXiv)

Numerous invariant (or equivariant) neural networks have succeeded in handling invariant data such as point clouds and graphs. However, a generalization theory for the neural networks has not been well developed, because several essential factors for the theory, such as network size and margin distribution, are not deeply connected to the invariance and equivariance. In this study, we develop a novel generalization error bound for invariant and equivariant deep neural networks. To describe the effect of invariance and equivariance on generalization, we develop a notion of a textitquotient feature space, which measures the effect of group actions for the properties. Our main result proves that the volume of quotient feature spaces can describe the generalization error. Furthermore, the bound shows that the invariance and equivariance significantly improve the leading term of the bound. We apply our result to specific invariant and equivariant networks, such as DeepSets (Zaheer et al. (2017)), and show that their generalization bound is considerably improved by √n!, where n! is the number of permutations. We also discuss the expressive power of invariant DNNs and show that they can achieve an optimal approximation rate. Our experimental result supports our theoretical claims.