# Stochastic gradient descent in high dimensions for multi-spiked tensor PCA

arXiv: https://arxiv.org/abs/2410.18162 · HTML: https://mlsanity.com/paper/2410.18162/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Oct 2024.

## Built on by (most cited first, 7 of 7)

- [The Generative Leap: Sharp Sample Complexity for Efficiently Learning Gaussian Multi-Index Models](https://mlsanity.com/paper/2506.05500/) (2025)
- [Deep Learning as Neural Low-Degree Filtering: A Spectral Theory of Hierarchical Feature Learning](https://mlsanity.com/paper/2605.13612/) (2026)
- [Permutation recovery of spikes in noisy high-dimensional tensor estimation](https://mlsanity.com/paper/2412.14650/) (2024)
- [Signal Recovery Using a Spiked Mixture Model](https://mlsanity.com/paper/2501.01840/) (2025)
- [Statistical Limits in Random Tensors with Multiple Correlated Spikes](https://mlsanity.com/paper/2503.03356/) (2025)
- [A Fourier perspective on the learning dynamics of neural networks: from sample complexities to mechanistic insights](https://mlsanity.com/paper/2605.16913/) (2026)
- [Non-asymptotic Tail Bounds for the Kostlan--Shub--Smale Field: Tensor PCA and Spherical k-Spin Complexity](https://mlsanity.com/paper/2606.17665/) (2026)

## Abstract (arXiv)

We study the high-dimensional dynamics of online stochastic gradient descent (SGD) for the multi-spiked tensor model. This multi-index model arises from the tensor principal component analysis (PCA) problem with multiple spikes, where the goal is to estimate r unknown signal vectors within the N-dimensional unit sphere through maximum likelihood estimation from noisy observations of a p-tensor. We determine the number of samples and the conditions on the signal-to-noise ratios (SNRs) required to efficiently recover the unknown spikes from natural random initializations. We show that full recovery of all spikes is possible provided a number of sample scaling as Np-2, matching the algorithmic threshold identified in the rank-one case [Ben Arous, Gheissari, Jagannath 2020, 2021]. Our results are obtained through a detailed analysis of a low-dimensional system that describes the evolution of the correlations between the estimators and the spikes, while controlling the noise in the dynamics. We find that the spikes are recovered sequentially in a process we term "sequential elimination": once a correlation exceeds a critical threshold, all correlations sharing a row or column index become sufficiently small, allowing the next correlation to grow and become macroscopic. The order in which correlations become macroscopic depends on their initial values and the corresponding SNRs, leading to either exact recovery or recovery of a permutation of the spikes. In the matrix case, when p=2, if the SNRs are sufficiently separated, we achieve exact recovery of the spikes, whereas equal SNRs lead to recovery of the subspace spanned by them.