Subspace Learning Machine (SLM): Methodology and Performance

Inspired by the feedforward multilayer perceptron (FF-MLP), decision tree (DT) and extreme learning machine (ELM), a new classification model, called the subspace learning machine (SLM), is proposed in this work. SLM first identifies a discriminant subspace, S0, by examining the discriminant power of each input feature. Then, it uses probabilistic projections of features in S0 to yield 1D subspaces and finds the optimal partition for each of them. This is equivalent to partitioning S0 with hyperplanes. A criterion is developed to choose the best q partitions that yield 2q partitioned subspaces among them. We assign S0 to the root node of a decision tree and the intersections of 2q subspaces to its child nodes of depth one. The partitioning process is recursively applied at each child node to build an SLM tree. When the samples at a child node are sufficiently pure, the partitioning process stops and each leaf node makes a prediction. The idea can be generalized to regression, leading to the subspace learning regressor (SLR). Furthermore, ensembles of SLM/SLR trees can yield a stronger predictor. Extensive experiments are conducted for performance benchmarking among SLM/SLR trees, ensembles and classical classifiers/regressors.