# Relief-Based Feature Selection: Introduction and Review

arXiv: https://arxiv.org/abs/1711.08421 · HTML: https://mlsanity.com/paper/1711.08421/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Nov 2017.

## Built on by (most cited first, 30 of 38)

- [Even if Explanations: Prior Work, Desiderata & Benchmarks for Semi-Factual XAI](https://mlsanity.com/paper/2301.11970/) (2023)
- [A Theoretical Analysis on Independence-driven Importance Weighting for Covariate-shift Generalization](https://mlsanity.com/paper/2111.02355/) (2021)
- [An overview of artificial intelligence techniques for diagnosis of Schizophrenia based on magnetic resonance imaging modalities: Methods, challenges, and future works](https://mlsanity.com/paper/2103.03081/) (2021)
- [Curse of Dimensionality for TSK Fuzzy Neural Networks: Explanation and Solutions](https://mlsanity.com/paper/2102.04271/) (2021)
- [Masked Imitation Learning: Discovering Environment-Invariant Modalities in Multimodal Demonstrations](https://mlsanity.com/paper/2209.07682/) (2022)
- [Automatic feature selection and weighting in molecular systems using Differentiable Information Imbalance](https://mlsanity.com/paper/2411.00851/) (2024)
- [Identifying Relevant Features of CSE-CIC-IDS2018 Dataset for the Development of an Intrusion Detection System](https://mlsanity.com/paper/2307.11544/) (2023)
- [Mixture of Complementary Agents for Robust LLM Ensemble](https://mlsanity.com/paper/2605.24048/) (2026)
- [Feature Selection on a Flare Forecasting Testbed: A Comparative Study of 24 Methods](https://mlsanity.com/paper/2109.14770/) (2021)
- [Iterative missing value imputation based on feature importance](https://mlsanity.com/paper/2311.08005/) (2023)
- [Cost-based feature selection for network model choice](https://mlsanity.com/paper/2101.07766/) (2021)
- [A novel adversarial learning strategy for medical image classification](https://mlsanity.com/paper/2206.11501/) (2022)
- [STREAMLINE: A Simple, Transparent, End-To-End Automated Machine Learning Pipeline Facilitating Data Analysis and Algorithm Comparison](https://mlsanity.com/paper/2206.12002/) (2022)
- [Gram-Schmidt Methods for Unsupervised Feature Extraction and Selection](https://mlsanity.com/paper/2311.09386/) (2023)
- [A novel feature selection framework for incomplete data](https://mlsanity.com/paper/2312.04171/) (2023)
- [Feature Selection via Robust Weighted Score for High Dimensional Binary Class-Imbalanced Gene Expression Data](https://mlsanity.com/paper/2401.12667/) (2024)
- [Dynamic feature selection in medical predictive monitoring by reinforcement learning](https://mlsanity.com/paper/2405.19729/) (2024)
- [Effect sizes as a statistical feature-selector-based learning to detect breast cancer](https://mlsanity.com/paper/2411.06868/) (2024)
- [Frugal Machine Learning for Energy-efficient, and Resource-aware Artificial Intelligence](https://mlsanity.com/paper/2506.01869/) (2025)
- [How Usable is Automated Feature Engineering for Tabular Data?](https://mlsanity.com/paper/2508.13932/) (2025)
- [Two novel feature selection algorithms based on crowding distance](https://mlsanity.com/paper/2105.05212/) (2021)
- [A Bytecode-based Approach for Smart Contract Classification](https://mlsanity.com/paper/2106.15497/) (2021)
- [A novel evaluation methodology for supervised Feature Ranking algorithms](https://mlsanity.com/paper/2207.04258/) (2022)
- [An extended method for Statistical Signal Characterization using moments and cumulants, as a fast and accurate pre-processing stage of simple ANNs applied to the recognition of pattern alterations in pulse-like waveforms](https://mlsanity.com/paper/2212.14783/) (2022)
- [Analogical Relevance Index](https://mlsanity.com/paper/2301.04134/) (2023)
- [Feature selection simultaneously preserving both class and cluster structures](https://mlsanity.com/paper/2307.03902/) (2023)
- [Genetic heterogeneity analysis using genetic algorithm and network science](https://mlsanity.com/paper/2308.06429/) (2023)
- [Hierarchical Ensemble-Based Feature Selection for Time Series Forecasting](https://mlsanity.com/paper/2310.17544/) (2023)
- [A Machine Learning Approach for Identifying Anatomical Biomarkers of Early Mild Cognitive Impairment](https://mlsanity.com/paper/2407.00040/) (2024)
- [Multi-Task Adversarial Variational Autoencoder for Estimating Biological Brain Age with Multimodal Neuroimaging](https://mlsanity.com/paper/2411.10100/) (2024)

## Abstract (arXiv)

Feature selection plays a critical role in biomedical data mining, driven by increasing feature dimensionality in target problems and growing interest in advanced but computationally expensive methodologies able to model complex associations. Specifically, there is a need for feature selection methods that are computationally efficient, yet sensitive to complex patterns of association, e.g. interactions, so that informative features are not mistakenly eliminated prior to downstream modeling. This paper focuses on Relief-based algorithms (RBAs), a unique family of filter-style feature selection algorithms that have gained appeal by striking an effective balance between these objectives while flexibly adapting to various data characteristics, e.g. classification vs. regression. First, this work broadly examines types of feature selection and defines RBAs within that context. Next, we introduce the original Relief algorithm and associated concepts, emphasizing the intuition behind how it works, how feature weights generated by the algorithm can be interpreted, and why it is sensitive to feature interactions without evaluating combinations of features. Lastly, we include an expansive review of RBA methodological research beyond Relief and its popular descendant, ReliefF. In particular, we characterize branches of RBA research, and provide comparative summaries of RBA algorithms including contributions, strategies, functionality, time complexity, adaptation to key data characteristics, and software availability.