# On Supervised Feature Selection from High Dimensional Feature Spaces

arXiv: https://arxiv.org/abs/2203.11924 · HTML: https://mlsanity.com/paper/2203.11924/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 43 references that are in the map.
- First posted on arXiv 22 Mar 2022.

## Builds on (most cited first, 11 of 11)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Interpretable Convolutional Neural Networks via Feedforward Design](https://mlsanity.com/paper/1810.02786/) (2018)
- [PointHop: An Explainable Machine Learning Method for Point Cloud Classification](https://mlsanity.com/paper/1907.12766/) (2019)
- [PixelHop++: A Small Successive-Subspace-Learning-Based (SSL-based) Model for Image Classification](https://mlsanity.com/paper/2002.03141/) (2020)
- [Survey on Feature Selection](https://mlsanity.com/paper/1510.02892/) (2015)
- [On Data-Driven Saak Transform](https://mlsanity.com/paper/1710.04176/) (2017)
- [PointHop++: A Lightweight Learning Model on Point Sets for 3D Classification](https://mlsanity.com/paper/2002.03281/) (2020)
- [VoxelHop: Successive Subspace Learning for ALS Disease Classification Using Structural MRI](https://mlsanity.com/paper/2101.05131/) (2021)
- [FaceHop: A Light-Weight Low-Resolution Face Gender Classification Method](https://mlsanity.com/paper/2007.09510/) (2020)
- [A Saak Transform Approach to Efficient, Scalable and Robust Handwritten Digits Recognition](https://mlsanity.com/paper/1710.10714/) (2017)
- [Visualization, Discriminability and Applications of Interpretable Saak Features](https://mlsanity.com/paper/1902.09107/) (2019)

## Built on by (most cited first, 23 of 23)

- [Green Learning: Introduction, Examples and Outlook](https://mlsanity.com/paper/2210.00965/) (2022)
- [Knowledge Graph Embedding: An Overview](https://mlsanity.com/paper/2309.12501/) (2023)
- [GreenKGC: A Lightweight Knowledge Graph Completion Method](https://mlsanity.com/paper/2208.09137/) (2022)
- [DefakeHop++: An Enhanced Lightweight Deepfake Detector](https://mlsanity.com/paper/2205.00211/) (2022)
- [Subspace Learning Machine (SLM): Methodology and Performance](https://mlsanity.com/paper/2205.05296/) (2022)
- [Design of Supervision-Scalable Learning Systems: Methodology and Performance Benchmarking](https://mlsanity.com/paper/2206.09061/) (2022)
- [S3I-PointHop: SO(3)-Invariant PointHop for 3D Point Cloud Classification](https://mlsanity.com/paper/2302.11506/) (2023)
- [Knowledge Graph Embedding with 3D Compound Geometric Transformations](https://mlsanity.com/paper/2304.00378/) (2023)
- [Enhancing Edge Intelligence with Highly Discriminant LNT Features](https://mlsanity.com/paper/2312.14968/) (2023)
- [Acceleration of Subspace Learning Machine via Particle Swarm Optimization and Parallel Processing](https://mlsanity.com/paper/2208.07023/) (2022)
- [GreenCOD: A Green Camouflaged Object Detection Method](https://mlsanity.com/paper/2405.16144/) (2024)
- [GUSL-Dehaze: A Green U-Shaped Learning Approach to Image Dehazing](https://mlsanity.com/paper/2510.20266/) (2025)
- [Green Steganalyzer: A Green Learning Approach to Image Steganalysis](https://mlsanity.com/paper/2306.04008/) (2023)
- [LSR: A Light-Weight Super-Resolution Method](https://mlsanity.com/paper/2302.13596/) (2023)
- [A Tiny Machine Learning Model for Point Cloud Object Classification](https://mlsanity.com/paper/2303.10898/) (2023)
- [Unsupervised Green Object Tracker (GOT) without Offline Pre-training](https://mlsanity.com/paper/2309.09078/) (2023)
- [GWPT: A Green Word-Embedding-based POS Tagger](https://mlsanity.com/paper/2401.07475/) (2024)
- [Word Embedding Dimension Reduction via Weakly-Supervised Feature Selection](https://mlsanity.com/paper/2407.12342/) (2024)
- [Efficient Human-Object-Interaction (EHOI) Detection via Interaction Label Coding and Conditional Decision](https://mlsanity.com/paper/2408.07018/) (2024)
- [Green Learning for STAR-RIS mmWave Systems with Implicit CSI](https://mlsanity.com/paper/2509.06820/) (2025)
- [LG-NuSegHop: A Local-to-Global Self-Supervised Pipeline For Nuclei Instance Segmentation](https://mlsanity.com/paper/2511.04892/) (2025)
- [A Green Learning Approach to LDCT Image Restoration](https://mlsanity.com/paper/2602.19540/) (2026)
- [GreenPhase: A Green Learning Approach for Earthquake Phase Picking](https://mlsanity.com/paper/2603.03344/) (2026)

## Abstract (arXiv)

The application of machine learning to image and video data often yields a high dimensional feature space. Effective feature selection techniques identify a discriminant feature subspace that lowers computational and modeling costs with little performance degradation. A novel supervised feature selection methodology is proposed for machine learning decisions in this work. The resulting tests are called the discriminant feature test (DFT) and the relevant feature test (RFT) for the classification and regression problems, respectively. The DFT and RFT procedures are described in detail. Furthermore, we compare the effectiveness of DFT and RFT with several classic feature selection methods. To this end, we use deep features obtained by LeNet-5 for MNIST and Fashion-MNIST datasets as illustrative examples. Other datasets with handcrafted and gene expressions features are also included for performance evaluation. It is shown by experimental results that DFT and RFT can select a lower dimensional feature subspace distinctly and robustly while maintaining high decision performance.