# ManiFeSt: Manifold-based Feature Selection for Small Data Sets

arXiv: https://arxiv.org/abs/2207.08574 · HTML: https://mlsanity.com/paper/2207.08574/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 75 references that are in the map.
- First posted on arXiv 18 Jul 2022.

## Builds on (most cited first, 8 of 8)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [A Review of Feature Selection Methods Based on Mutual Information](https://mlsanity.com/paper/1509.07577/) (2015)
- [Locally Sparse Neural Networks for Tabular Biomedical Data](https://mlsanity.com/paper/2106.06468/) (2021)
- [Maximum Relevance and Minimum Redundancy Feature Selection Methods for a Marketing Machine Learning Platform](https://mlsanity.com/paper/1908.05376/) (2019)
- [Doubly-Stochastic Normalization of the Gaussian Kernel is Robust to Heteroskedastic Noise](https://mlsanity.com/paper/2006.00402/) (2020)
- [A Brain-inspired Algorithm for Training Highly Sparse Neural Networks](https://mlsanity.com/paper/1903.07138/) (2019)
- [Spectral Embedding Norm: Looking Deep into the Spectrum of the Graph Laplacian](https://mlsanity.com/paper/1810.10695/) (2018)
- [Spatiotemporal Analysis Using Riemannian Composition of Diffusion Operators](https://mlsanity.com/paper/2201.08530/) (2022)

## Built on by (most cited first, 2 of 2)

- [Multi-modal Differentiable Unsupervised Feature Selection](https://mlsanity.com/paper/2303.09381/) (2023)
- [DiSC: Differential Spectral Clustering of Features](https://mlsanity.com/paper/2211.05314/) (2022)

## Abstract (arXiv)

In this paper, we present a new method for few-sample supervised feature selection (FS). Our method first learns the manifold of the feature space of each class using kernels capturing multi-feature associations. Then, based on Riemannian geometry, a composite kernel is computed, extracting the differences between the learned feature associations. Finally, a FS score based on spectral analysis is proposed. Considering multi-feature associations makes our method multivariate by design. This in turn allows for the extraction of the hidden manifold underlying the features and avoids overfitting, facilitating few-sample FS. We showcase the efficacy of our method on illustrative examples and several benchmarks, where our method demonstrates higher accuracy in selecting the informative features compared to competing methods. In addition, we show that our FS leads to improved classification and better generalization when applied to test data.