# Supervised Dimensionality Reduction for Big Data

arXiv: https://arxiv.org/abs/1709.01233 · HTML: https://mlsanity.com/paper/1709.01233/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Sep 2017.

## Built on by (most cited first, 13 of 13)

- [Interpreting the Curse of Dimensionality from Distance Concentration and Manifold Effect](https://mlsanity.com/paper/2401.00422/) (2023)
- [Simultaneous Dimensionality Reduction: A Data Efficient Approach for Multimodal Representations Learning](https://mlsanity.com/paper/2310.04458/) (2023)
- [SCALAR: Self-Calibrating Adaptive Latent Attention Representation Learning](https://mlsanity.com/paper/2510.16474/) (2025)
- [Random Forest Autoencoders for Guided Representation Learning](https://mlsanity.com/paper/2502.13257/) (2025)
- [Learning effective dynamics from data-driven stochastic systems](https://mlsanity.com/paper/2205.04151/) (2022)
- [An Additive Autoencoder for Dimension Estimation](https://mlsanity.com/paper/2210.06773/) (2022)
- [Sampling-enabled scalable manifold learning unveils the discriminative cluster structure of high-dimensional data](https://mlsanity.com/paper/2401.01100/) (2024)
- [Contrastive inverse regression for dimension reduction](https://mlsanity.com/paper/2305.12287/) (2023)
- [Gradient Boosting Mapping for Dimensionality Reduction and Feature Extraction](https://mlsanity.com/paper/2405.08486/) (2024)
- [Using Slisemap to interpret physical data](https://mlsanity.com/paper/2310.15610/) (2023)
- [Synergistic eigenanalysis of covariance and Hessian matrices for enhanced binary classification](https://mlsanity.com/paper/2402.09281/) (2024)
- [Discriminative Subspace Emersion from learning feature relevances across different populations](https://mlsanity.com/paper/2504.00176/) (2025)
- [PEARL: Prototype-Enhanced Alignment for Label-Efficient Representation Learning with Deployment-Driven Insights from Digital Governance Communication Systems](https://mlsanity.com/paper/2601.17495/) (2026)

## Abstract (arXiv)

To solve key biomedical problems, experimentalists now routinely measure millions or billions of features (dimensions) per sample, with the hope that data science techniques will be able to build accurate data-driven inferences. Because sample sizes are typically orders of magnitude smaller than the dimensionality of these data, valid inferences require finding a low-dimensional representation that preserves the discriminating information (e.g., whether the individual suffers from a particular disease). There is a lack of interpretable supervised dimensionality reduction methods that scale to millions of dimensions with strong statistical theoretical guarantees.We introduce an approach, XOX, to extending principal components analysis by incorporating class-conditional moment estimates into the low-dimensional projection. The simplest ver-sion, "Linear Optimal Low-rank" projection (LOL), incorporates the class-conditional means. We prove, and substantiate with both synthetic and real data benchmarks, that LOL and its generalizations in the XOX framework lead to improved data representations for subsequent classification, while maintaining computational efficiency and scalability. Using multiple brain imaging datasets consisting of >150 million features, and several genomics datasets with>500,000 features, LOL outperforms other scalable linear dimensionality reduction techniques in terms of accuracy, while only requiring a few minutes on a standard desktop computer.