# Random-projection ensemble classification

arXiv: https://arxiv.org/abs/1504.04595 · HTML: https://mlsanity.com/paper/1504.04595/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Apr 2015.

## Built on by (most cited first, 29 of 29)

- [A survey of unsupervised learning methods for high-dimensional uncertainty quantification in black-box-type problems](https://mlsanity.com/paper/2202.04648/) (2022)
- [RPMixer: Shaking Up Time Series Forecasting with Random Projections for Large Spatial-Temporal Data](https://mlsanity.com/paper/2402.10487/) (2024)
- [Layer Adaptive Node Selection in Bayesian Neural Networks: Statistical Guarantees and Implementation Details](https://mlsanity.com/paper/2108.11000/) (2021)
- [Learning Augmentation Distributions using Transformed Risk Minimization](https://mlsanity.com/paper/2111.08190/) (2021)
- [An Introduction to Johnson-Lindenstrauss Transforms](https://mlsanity.com/paper/2103.00564/) (2021)
- [Johnson-Lindenstrauss Lemma, Linear and Nonlinear Random Projections, Random Fourier Features, and Random Kitchen Sinks: Tutorial and Survey](https://mlsanity.com/paper/2108.04172/) (2021)
- [Learning from Hypervectors: A Survey on Hypervector Encoding](https://mlsanity.com/paper/2308.00685/) (2023)
- [Are Ensembles Getting Better all the Time?](https://mlsanity.com/paper/2311.17885/) (2023)
- [RaSE: A Variable Screening Framework via Random Subspace Ensembles](https://mlsanity.com/paper/2102.03892/) (2021)
- [Sharp-SSL: Selective high-dimensional axis-aligned random projections for semi-supervised learning](https://mlsanity.com/paper/2304.09154/) (2023)
- [On the Rashomon ratio of infinite hypothesis sets](https://mlsanity.com/paper/2404.17746/) (2024)
- [Statistical optimality conditions for compressive ensembles](https://mlsanity.com/paper/2106.01092/) (2021)
- [Approximation and generalization properties of the random projection classification method](https://mlsanity.com/paper/2108.06339/) (2021)
- [On unsupervised projections and second order signals](https://mlsanity.com/paper/2204.05139/) (2022)
- [Dimension Reduction and MARS](https://mlsanity.com/paper/2302.05790/) (2023)
- [EnsLoss: Stochastic Calibrated Loss Ensembles for Preventing Overfitting in Classification](https://mlsanity.com/paper/2409.00908/) (2024)
- [TEC: Tensor Ensemble Classifier for Big Data](https://mlsanity.com/paper/2103.00025/) (2021)
- [Statistical embedding: Beyond principal components](https://mlsanity.com/paper/2106.01858/) (2021)
- [Invariance principle of random projection for the norm](https://mlsanity.com/paper/2112.00300/) (2021)
- [Exact Non-Oblivious Performance of Rademacher Random Embeddings](https://mlsanity.com/paper/2303.11774/) (2023)
- [Node Similarities under Random Projections: Limits and Pathological Cases](https://mlsanity.com/paper/2404.10148/) (2024)
- [Random-projection ensemble dimension reduction](https://mlsanity.com/paper/2410.04922/) (2024)
- [Conjunction Subspaces Test for Conformal and Selective Classification](https://mlsanity.com/paper/2410.12297/) (2024)
- [Oblique Bayesian additive regression trees](https://mlsanity.com/paper/2411.08849/) (2024)
- [Bags of Projected Nearest Neighbours: Competitors to Random Forests?](https://mlsanity.com/paper/2503.09651/) (2025)
- [Centroid Decision Forest](https://mlsanity.com/paper/2503.19306/) (2025)
- [ArrowFlow: Hierarchical Machine Learning in the Space of Permutations](https://mlsanity.com/paper/2604.04087/) (2026)
- [2D Stability Selection: Design Jittering for Doubly Stable Feature Selection](https://mlsanity.com/paper/2605.02205/) (2026)
- [Exact Limits of Random Projections for Preserving Geometry: Distance Recovery, Nearest-Neighbor Rankings, and Covariance Shape in Gaussian Models](https://mlsanity.com/paper/2609.02155/) (2026)

## Abstract (arXiv)

We introduce a very general method for high-dimensional classification, based on careful combination of the results of applying an arbitrary base classifier to random projections of the feature vectors into a lower-dimensional space. In one special case that we study in detail, the random projections are divided into disjoint groups, and within each group we select the projection yielding the smallest estimate of the test error. Our random projection ensemble classifier then aggregates the results of applying the base classifier on the selected projections, with a data-driven voting threshold to determine the final assignment. Our theoretical results elucidate the effect on performance of increasing the number of projections. Moreover, under a boundary condition implied by the sufficient dimension reduction assumption, we show that the test excess risk of the random projection ensemble classifier can be controlled by terms that do not depend on the original data dimension and a term that becomes negligible as the number of projections increases. The classifier is also compared empirically with several other popular high-dimensional classifiers via an extensive simulation study, which reveals its excellent finite-sample performance.