# Symmetry-Aware Feature Learning: A Polynomial Separation for Multi-Index Models

arXiv: https://arxiv.org/abs/2610.08420 · HTML: https://mlsanity.com/paper/2610.08420/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 2 of its 38 references that are in the map.
- First posted on arXiv 6 Oct 2026.

## Builds on (most cited first, 2 of 2)

- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [On the Benefits of Invariance in Neural Networks](https://mlsanity.com/paper/2005.00178/) (2020)

## Abstract (arXiv)

We establish a polynomial sample complexity separation between symmetry-aware and symmetry-agnostic feature learning. We study growing-rank multi-index models with high-dimensional Gaussian covariates in ℝd and r=Theta(dδ) teacher directions forming a cyclic symmetry orbit, where 0<δ<1/2. We compare three ways of exploiting this structure: architectural weight sharing, data augmentation over the full symmetry group, and learning without access to the symmetry. In particular, we analyze a symmetry-tied convolutional network, an untied network, and the same untied network trained with full-group data augmentation, using spherical online SGD with correlation loss. For a class of polynomial links with information exponent p≥3, we prove matching sample complexity bounds up to logarithmic factors: the tied and augmented learners achieve weak directional recovery in widetildeTheta(dp-1) samples, whereas the symmetry-agnostic learner requires widetildeTheta(rdp-1). For the pure quadratic Hermite link, the same separation holds for weak recovery of the teacher subspace, with sample complexities widetildeTheta(d) and widetildeTheta(rd), respectively. Thus, full-group data augmentation matches the sample efficiency of architectural weight sharing, and both provide a polynomial advantage over training without symmetry. For p≥3, the proof reveals a two-stage mechanism: fluctuations at initialization select one direction in the teacher orbit, after which localized growth amplifies its overlap to the weak recovery scale while competing overlaps remain near their initialization scale.