# Classification of high-dimensional data with spiked covariance matrix structure

arXiv: https://arxiv.org/abs/2110.01950 · HTML: https://mlsanity.com/paper/2110.01950/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 79 references that are in the map.
- First posted on arXiv 5 Oct 2021.

## Builds on (most cited first, 13 of 13)

- [Optimal whitening and decorrelation](https://mlsanity.com/paper/1512.00809/) (2015)
- [Rate-Optimal Perturbation Bounds for Singular Subspaces with Applications to High-Dimensional Statistics](https://mlsanity.com/paper/1605.00353/) (2016)
- [High-dimensional Linear Discriminant Analysis: Optimality, Adaptive Algorithm, and Missing Data](https://mlsanity.com/paper/1804.03018/) (2018)
- [Robust high dimensional factor models with applications to statistical machine learning](https://mlsanity.com/paper/1808.03889/) (2018)
- [A Convex Optimization Approach to High-Dimensional Sparse Quadratic Discriminant Analysis](https://mlsanity.com/paper/1912.02872/) (2019)
- [A Direct Approach for Sparse Quadratic Discriminant Analysis](https://mlsanity.com/paper/1510.00084/) (2015)
- [Multiclass Sparse Discriminant Analysis](https://mlsanity.com/paper/1504.05845/) (2015)
- [Sparsifying the Fisher Linear Discriminant by Rotation](https://mlsanity.com/paper/1408.4475/) (2014)
- [Significance testing in non-sparse high-dimensional linear models](https://mlsanity.com/paper/1610.02122/) (2016)
- [Canonical thresholding for non-sparse high-dimensional linear regression](https://mlsanity.com/paper/2007.12313/) (2020)
- [High-Dimensional Quadratic Discriminant Analysis under Spiked Covariance Model](https://mlsanity.com/paper/2006.14325/) (2020)
- [A New Reduced-Rank Linear Discriminant Analysis Method and Its Applications](https://mlsanity.com/paper/1511.00282/) (2015)
- [Optimal Covariance Estimation for Condition Number Loss in the Spiked Model](https://mlsanity.com/paper/1810.07403/) (2018)

## Abstract (arXiv)

We study the classification problem for high-dimensional data with n observations on p features where the p × p covariance matrix Sigma exhibits a spiked eigenvalue structure and the vector zeta, given by the difference between the em whitened mean vectors, is sparse. We analyze an adaptive classifier (adaptive with respect to the sparsity s) that first performs dimension reduction on the feature vectors prior to classification in the dimensionally reduced space, i.e., the classifier whitens the data, then screens the features by keeping only those corresponding to the s largest coordinates of zeta and finally applies Fisher linear discriminant on the selected features. Leveraging recent results on entrywise matrix perturbation bounds for covariance matrices, we show that the resulting classifier is Bayes optimal whenever n rightarrow ∞ and s √n-1 ln p rightarrow 0. Notably, our theory also guarantees Bayes optimality for the corresponding quadratic discriminant analysis (QDA). Experimental results on real and synthetic data further indicate that the proposed approach is competitive with state-of-the-art methods while operating on a substantially lower-dimensional representation.