# Sequential subspace methods on Stiefel manifold optimization

arXiv: https://arxiv.org/abs/2404.13301 · HTML: https://mlsanity.com/paper/2404.13301/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Apr 2024.

## Built on by (most cited first, 1 of 1)

- [Provable Model-Parallel Distributed Principal Component Analysis with Parallel Deflation](https://mlsanity.com/paper/2502.17615/) (2025)

## Abstract (arXiv)

We investigate the minimization of a quadratic function over Stiefel manifolds (the set of all orthogonal r- frames in Rn), which has applications in high-dimensional semi-supervised classification tasks. To reduce the computational complexity, we employ sequential subspace methods(SSM) to transform the high-dimensional problem to a series of low-dimensional ones. In this paper, our goal is to achieve an optimal solution of high quality, referred to as a ''qualified critical point". Qualified critical points are defined as those where the associated multiplier matrix meets specific upper-bound conditions. These points exhibit near-global optimality in quadratic optimization problems. In the context of a general quadratic, SSM generates a sequence of qualified critical points through low-dimensional surrogate regularized models. The convergence to a qualified critical point is guaranteed, when each SSM subspace is constructed from the following vectors: (i) a set of orthogonal unit vectors associated with the current iterate, (ii) a set of vectors representing the gradient of the objective, and (iii) a set of eigenvectors links to the smallest r eigenvalues of the system matrix. Furthermore, incorporating Newton direction vectors into the subspaces can significantly accelerate the convergence of SSM.