# Sparse GCA and Thresholded Gradient Descent

arXiv: https://arxiv.org/abs/2107.00371 · HTML: https://mlsanity.com/paper/2107.00371/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 44 references that are in the map.
- First posted on arXiv 1 Jul 2021.

## Builds on (most cited first, 10 of 10)

- [Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overview](https://mlsanity.com/paper/1809.09573/) (2018)
- [No Spurious Local Minima in Nonconvex Low Rank Problems: A Unified Geometric Analysis](https://mlsanity.com/paper/1704.00708/) (2017)
- [Low-rank Solutions of Linear Matrix Equations via Procrustes Flow](https://mlsanity.com/paper/1507.03566/) (2015)
- [Solving Random Quadratic Systems of Equations Is Nearly as Easy as Solving Linear Systems](https://mlsanity.com/paper/1505.05114/) (2015)
- [Sparse CCA: Adaptive Estimation and Computational Barriers](https://mlsanity.com/paper/1409.8565/) (2014)
- [Estimating Number of Factors by Adjusted Eigenvalues Thresholding](https://mlsanity.com/paper/1909.10710/) (2019)
- [Computational Lower Bounds for Sparse PCA](https://mlsanity.com/paper/1304.0828/) (2013)
- [Generalized Canonical Correlation Analysis for Classification](https://mlsanity.com/paper/1304.7981/) (2013)
- [Sparse Generalized Eigenvalue Problem: Optimal Statistical Rates via Truncated Rayleigh Flow](https://mlsanity.com/paper/1604.08697/) (2016)
- [Sparse CCA via Precision Adjusted Iterative Thresholding](https://mlsanity.com/paper/1311.6186/) (2013)

## Built on by (most cited first, 4 of 4)

- [Safeguarding Data in Multimodal AI: A Differentially Private Approach to CLIP Training](https://mlsanity.com/paper/2306.08173/) (2023)
- [ℓ1-norm constrained multi-block sparse canonical correlation analysis via proximal gradient descent](https://mlsanity.com/paper/2201.05289/) (2022)
- [Differentially Private Sliced Inverse Regression: Minimax Optimality and Algorithm](https://mlsanity.com/paper/2401.08150/) (2024)
- [A Double Tracking Method for Optimization with Decentralized Generalized Orthogonality Constraints](https://mlsanity.com/paper/2409.04998/) (2024)

## Abstract (arXiv)

Generalized correlation analysis (GCA) is concerned with uncovering linear relationships across multiple datasets. It generalizes canonical correlation analysis that is designed for two datasets. We study sparse GCA when there are potentially multiple generalized correlation tuples in data and the loading matrix has a small number of nonzero rows. It includes sparse CCA and sparse PCA of correlation matrices as special cases. We first formulate sparse GCA as generalized eigenvalue problems at both population and sample levels via a careful choice of normalization constraints. Based on a Lagrangian form of the sample optimization problem, we propose a thresholded gradient descent algorithm for estimating GCA loading vectors and matrices in high dimensions. We derive tight estimation error bounds for estimators generated by the algorithm with proper initialization. We also demonstrate the prowess of the algorithm on a number of synthetic datasets.