# Diffusion Approximations for Online Principal Component Estimation and Global Convergence

arXiv: https://arxiv.org/abs/1808.09645 · HTML: https://mlsanity.com/paper/1808.09645/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Aug 2018.

## Built on by (most cited first, 3 of 3)

- [High-dimensional limit theorems for SGD: Effective dynamics and critical scaling](https://mlsanity.com/paper/2206.04030/) (2022)
- [Limit Theorems for Stochastic Gradient Descent in High-Dimensional Single-Layer Networks](https://mlsanity.com/paper/2511.02258/) (2025)
- [A General Continuous-Time Formulation of Stochastic ADMM and Its Variants](https://mlsanity.com/paper/2404.14358/) (2024)

## Abstract (arXiv)

In this paper, we propose to adopt the diffusion approximation tools to study the dynamics of Oja's iteration which is an online stochastic gradient descent method for the principal component analysis. Oja's iteration maintains a running estimate of the true principal component from streaming data and enjoys less temporal and spatial complexities. We show that the Oja's iteration for the top eigenvector generates a continuous-state discrete-time Markov chain over the unit sphere. We characterize the Oja's iteration in three phases using diffusion approximation and weak convergence tools. Our three-phase analysis further provides a finite-sample error bound for the running estimate, which matches the minimax information lower bound for principal component analysis under the additional assumption of bounded samples.