# Sparsifying the Fisher Linear Discriminant by Rotation

arXiv: https://arxiv.org/abs/1408.4475 · HTML: https://mlsanity.com/paper/1408.4475/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Aug 2014.

## Built on by (most cited first, 6 of 6)

- [Towards Understanding the Impact of Model Size on Differential Private Classification](https://mlsanity.com/paper/2111.13895/) (2021)
- [Towards Demystifying the Generalization Behaviors When Neural Collapse Emerges](https://mlsanity.com/paper/2310.08358/) (2023)
- [Neural Multivariate Regression: Qualitative Insights from the Unconstrained Feature Model](https://mlsanity.com/paper/2505.09308/) (2025)
- [The effect of whitening on explanation performance](https://mlsanity.com/paper/2602.09278/) (2026)
- [Classification of high-dimensional data with spiked covariance matrix structure](https://mlsanity.com/paper/2110.01950/) (2021)
- [From Kinetic Theory to AI: a Rediscovery of High-Dimensional Divergences and Their Properties](https://mlsanity.com/paper/2507.11387/) (2025)

## Abstract (arXiv)

Many high dimensional classification techniques have been proposed in the literature based on sparse linear discriminant analysis (LDA). To efficiently use them, sparsity of linear classifiers is a prerequisite. However, this might not be readily available in many applications, and rotations of data are required to create the needed sparsity. In this paper, we propose a family of rotations to create the required sparsity. The basic idea is to use the principal components of the sample covariance matrix of the pooled samples and its variants to rotate the data first and to then apply an existing high dimensional classifier. This rotate-and-solve procedure can be combined with any existing classifiers, and is robust against the sparsity level of the true model. We show that these rotations do create the sparsity needed for high dimensional classifications and provide theoretical understanding why such a rotation works empirically. The effectiveness of the proposed method is demonstrated by a number of simulated and real data examples, and the improvements of our method over some popular high dimensional classification rules are clearly shown.