# Filter Methods for Feature Selection in Supervised Machine Learning Applications -- Review and Benchmark

arXiv: https://arxiv.org/abs/2111.12140 · HTML: https://mlsanity.com/paper/2111.12140/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 105 references that are in the map.
- First posted on arXiv 23 Nov 2021.

## Builds on (most cited first, 4 of 4)

- [Deep Learning: A Critical Appraisal](https://mlsanity.com/paper/1801.00631/) (2018)
- [Deep learning in business analytics and operations research: Models, applications and managerial implications](https://mlsanity.com/paper/1806.10897/) (2018)
- [Cost-Sensitive Feature Selection by Optimizing F-Measures](https://mlsanity.com/paper/1904.02301/) (2019)
- [A Taxonomy of Data Quality Challenges in Empirical Software Engineering](https://mlsanity.com/paper/2106.06141/) (2021)

## Built on by (most cited first, 4 of 4)

- [Automatic feature selection and weighting in molecular systems using Differentiable Information Imbalance](https://mlsanity.com/paper/2411.00851/) (2024)
- [Adversarial Learning for Feature Shift Detection and Correction](https://mlsanity.com/paper/2312.04546/) (2023)
- [ROOFS: RObust biOmarker Feature Selection](https://mlsanity.com/paper/2601.05151/) (2026)
- [Feature Shift Localization Network](https://mlsanity.com/paper/2506.09101/) (2025)

## Abstract (arXiv)

The amount of data for machine learning (ML) applications is constantly growing. Not only the number of observations, especially the number of measured variables (features) increases with ongoing digitization. Selecting the most appropriate features for predictive modeling is an important lever for the success of ML applications in business and research. Feature selection methods (FSM) that are independent of a certain ML algorithm - so-called filter methods - have been numerously suggested, but little guidance for researchers and quantitative modelers exists to choose appropriate approaches for typical ML problems. This review synthesizes the substantial literature on feature selection benchmarking and evaluates the performance of 58 methods in the widely used R environment. For concrete guidance, we consider four typical dataset scenarios that are challenging for ML models (noisy, redundant, imbalanced data and cases with more features than observations). Drawing on the experience of earlier benchmarks, which have considered much fewer FSMs, we compare the performance of the methods according to four criteria (predictive performance, number of relevant features selected, stability of the feature sets and runtime). We found methods relying on the random forest approach, the double input symmetrical relevance filter (DISR) and the joint impurity filter (JIM) were well-performing candidate methods for the given dataset scenarios.