mlsanityData to 7 Oct 2026

Efficient Estimation of the Central Mean Subspace via Smoothed Gradient Outer Products

arXiv 2312.15469 ↗First posted 24 Dec 2023stat.ML cs.LG stat.MEcited by 7

Not on the map. Why?

Abstract (arXiv)

We consider the problem of sufficient dimension reduction (SDR) for multi-index models. The estimators of the central mean subspace in prior works either have slow (non-parametric) convergence rates, or rely on stringent distributional conditions (e.g., the covariate distribution PX being elliptical symmetric). In this paper, we show that a fast parametric convergence rate of form Cd · n-1/2 is achievable via estimating the expected smoothed gradient outer product, for a general class of distribution PX admitting Gaussian or heavier distributions. When the link function is a polynomial with a degree of at most r and PX is the standard Gaussian, we show that the prefactor depends on the ambient dimension d as Cd propto dr.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 9 of its 36 references that are in the map. First posted on arXiv 24 Dec 2023.

Builds on

9 of 36 references.

References · 9

  1. 2020Provable Meta-Learning of Linear Representationscited by 161
  2. 2022Neural Networks can Learn Representations with Gradient Descentcited by 161
  3. 2020Spectral Methods for Data Science: A Statistical Perspectivecited by 111
  4. 2022Learning Single-Index Models with Shallow Neural Networkscited by 92
  5. 2022Neural Networks Efficiently Learn Low-Dimensional Representations with SGDcited by 61
  6. 2023Smoothing the Landscape Boosts the Signal for SGD: Optimal Sample Complexity for Learning Single Index Modelscited by 60
  7. 2013Loss minimization and parameter estimation with heavy tailscited by 50
  8. 2020Algorithms and SQ Lower Bounds for PAC Learning One-Hidden-Layer ReLU Networkscited by 49
  9. 2020Learning Polynomials of Few Relevant Dimensionscited by 34

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 1; 2026: 2.

7 papers.

  1. 2024Emergence in non-neural models: grokking modular arithmetic via average gradient outer productcited by 26
  2. 2024Average gradient outer product as a mechanism for deep neural collapsecited by 21
  3. 2025Survey on Algorithms for multi-index modelscited by 15
  4. 2024Feature learning as alignment: a structural property of gradient descent in non-linear neural networkscited by 9
  5. 2024Robust Feature Learning for Multi-Index Models in High Dimensionscited by 5
  6. 2026When More Data Doesn't Help: Limits of Adaptation in Multitask Learningcited by 1
  7. 2026Local EGOP for Continuous Index Learningcited by 0