O(k)-Equivariant Dimensionality Reduction on Stiefel Manifolds

Many real-world datasets live on high-dimensional Stiefel and Grassmannian manifolds, Vk(ℝN) and Gr(k, ℝN) respectively, and benefit from projection onto lower-dimensional Stiefel and Grassmannian manifolds. In this work, we propose an algorithm called textitPrincipal Stiefel Coordinates (PSC) to reduce data dimensionality from Vk(ℝN) to Vk(ℝn) in an textitO(k)-equivariant manner (k ≤ n ll N). We begin by observing that each element α ∈ Vn(ℝN) defines an isometric embedding of Vk(ℝn) into Vk(ℝN). Next, we describe two ways of finding a suitable embedding map α: one via an extension of principal component analysis (αPCA), and one that further minimizes data fit error using gradient descent (αGD). Then, we define a continuous and O(k)-equivariant map πα that acts as a "closest point operator" to project the data onto the image of Vk(ℝn) in Vk(ℝN) under the embedding determined by α, while minimizing distortion. Because this dimensionality reduction is O(k)-equivariant, these results extend to Grassmannian manifolds as well. Lastly, we show that π_αPCA globally minimizes projection error in a noiseless setting, while π_αGD achieves a meaningfully different and improved outcome when the data does not lie exactly on the image of a linearly embedded lower-dimensional Stiefel manifold as above. Multiple numerical experiments using synthetic and real-world data are performed.