# Subspace Learning Machine (SLM): Methodology and Performance

arXiv: https://arxiv.org/abs/2205.05296 · HTML: https://mlsanity.com/paper/2205.05296/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 56 references that are in the map.
- First posted on arXiv 11 May 2022.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Interpretable Convolutional Neural Networks via Feedforward Design](https://mlsanity.com/paper/1810.02786/) (2018)
- [PixelHop++: A Small Successive-Subspace-Learning-Based (SSL-based) Model for Image Classification](https://mlsanity.com/paper/2002.03141/) (2020)
- [Understanding Convolutional Neural Networks with A Mathematical Model](https://mlsanity.com/paper/1609.04112/) (2016)
- [On Supervised Feature Selection from High Dimensional Feature Spaces](https://mlsanity.com/paper/2203.11924/) (2022)
- [On Data-Driven Saak Transform](https://mlsanity.com/paper/1710.04176/) (2017)
- [A Saak Transform Approach to Efficient, Scalable and Robust Handwritten Digits Recognition](https://mlsanity.com/paper/1710.10714/) (2017)
- [From Two-Class Linear Discriminant Analysis to Interpretable Multilayer Perceptron Design](https://mlsanity.com/paper/2009.04442/) (2020)
- [Interpretable Locally Adaptive Nearest Neighbors](https://mlsanity.com/paper/2011.03904/) (2020)

## Built on by (most cited first, 4 of 4)

- [Green Learning: Introduction, Examples and Outlook](https://mlsanity.com/paper/2210.00965/) (2022)
- [Enhancing Edge Intelligence with Highly Discriminant LNT Features](https://mlsanity.com/paper/2312.14968/) (2023)
- [Acceleration of Subspace Learning Machine via Particle Swarm Optimization and Parallel Processing](https://mlsanity.com/paper/2208.07023/) (2022)
- [Discriminative Subspace Emersion from learning feature relevances across different populations](https://mlsanity.com/paper/2504.00176/) (2025)

## Abstract (arXiv)

Inspired by the feedforward multilayer perceptron (FF-MLP), decision tree (DT) and extreme learning machine (ELM), a new classification model, called the subspace learning machine (SLM), is proposed in this work. SLM first identifies a discriminant subspace, S0, by examining the discriminant power of each input feature. Then, it uses probabilistic projections of features in S0 to yield 1D subspaces and finds the optimal partition for each of them. This is equivalent to partitioning S0 with hyperplanes. A criterion is developed to choose the best q partitions that yield 2q partitioned subspaces among them. We assign S0 to the root node of a decision tree and the intersections of 2q subspaces to its child nodes of depth one. The partitioning process is recursively applied at each child node to build an SLM tree. When the samples at a child node are sufficiently pure, the partitioning process stops and each leaf node makes a prediction. The idea can be generalized to regression, leading to the subspace learning regressor (SLR). Furthermore, ensembles of SLM/SLR trees can yield a stronger predictor. Extensive experiments are conducted for performance benchmarking among SLM/SLR trees, ensembles and classical classifiers/regressors.