# Online Supervised Dimension Reduction with Random Features: Diagnostics and Computational Trade-offs

arXiv: https://arxiv.org/abs/2609.20454 · HTML: https://mlsanity.com/paper/2609.20454/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 44 references that are in the map.
- First posted on arXiv 17 Sep 2026.

## Builds on (most cited first, 11 of 11)

- [Riemannian Adaptive Optimization Methods](https://mlsanity.com/paper/1810.00760/) (2018)
- [Streaming Kernel PCA with tildeO(√n) Random Features](https://mlsanity.com/paper/1808.00934/) (2018)
- [Supervised Dimensionality Reduction and Visualization using Centroid-encoder](https://mlsanity.com/paper/2002.11934/) (2020)
- [Optimization without Retraction on the Random Generalized Stiefel Manifold](https://mlsanity.com/paper/2405.01702/) (2024)
- [Robust Streaming PCA](https://mlsanity.com/paper/1902.03223/) (2019)
- [Streaming Kernel PCA Algorithm With Small Space](https://mlsanity.com/paper/2303.04555/) (2023)
- [A general framework of Riemannian adaptive optimization methods with a convergence analysis](https://mlsanity.com/paper/2409.00859/) (2024)
- [Online Supervised Subspace Tracking](https://mlsanity.com/paper/1509.00137/) (2015)
- [SRP: Efficient class-aware embedding learning for large-scale data via supervised random projections](https://mlsanity.com/paper/1811.03166/) (2018)
- [Covariance-free Partial Least Squares: An Incremental Dimensionality Reduction Method](https://mlsanity.com/paper/1910.02319/) (2019)
- [Online Kernel Sliced Inverse Regression](https://mlsanity.com/paper/2301.09516/) (2023)

## Abstract (arXiv)

Accurate optimization of a supervised spectral objective need not produce an accurate population subspace or a better predictive representation. We investigate these distinctions for Online Kernel Supervised Principal Component Analysis (OKSPCA), which combines a centered cross-moment in finite random-feature coordinates with an Adam-style orthonormal basis update for an established objective. Fixed-map consistency, concentration and perturbation results describe the estimator and its exact subspace; same-target comparisons then assess the practical iterate separately. Across six predictive benchmarks, performance depends on the declared pipeline: replacing the tracker with the exact empirical target leaves the two regression deficits largely unchanged. Direct classification-rank models capture nearly all terminal objective energy on average, but a saved intermediate state exhibits substantial geometric deviation; a controlled sample-size study further separates empirical accuracy from population recovery. In distinct numerical-service workloads, exact on-request computation is faster in the tested classification settings, whereas Adam saves time relative to the tested full thin-SVD service for some dense wider-regression requests, alongside persistent geometric error. These diagnostics limit explanations based solely on terminal optimization accuracy and distinguish numerical cost from quality, rank coverage and freshness; they establish neither practical-tracker convergence nor predictive or deployment benefits from basis availability.