# Leveraging Model Inherent Variable Importance for Stable Online Feature Selection

arXiv: https://arxiv.org/abs/2006.10398 · HTML: https://mlsanity.com/paper/2006.10398/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jun 2020.

## Built on by (most cited first, 9 of 9)

- [Deep Neural Networks and Tabular Data: A Survey](https://mlsanity.com/paper/2110.01889/) (2021)
- [On Baselines for Local Feature Attributions](https://mlsanity.com/paper/2101.00905/) (2021)
- [Change Detection for Local Explainability in Evolving Data Streams](https://mlsanity.com/paper/2209.02764/) (2022)
- [Dynamic Model Tree for Interpretable Data Stream Learning](https://mlsanity.com/paper/2203.16181/) (2022)
- [Standardized Evaluation of Machine Learning Methods for Evolving Data Streams](https://mlsanity.com/paper/2204.13625/) (2022)
- [Employing Two-Dimensional Word Embedding for Difficult Tabular Data Stream Classification](https://mlsanity.com/paper/2404.15836/) (2024)
- [Online Feature Screening for Data Streams with Concept Drift](https://mlsanity.com/paper/2104.02883/) (2021)
- [Online Feature Selection for Efficient Learning in Networked Systems](https://mlsanity.com/paper/2112.08253/) (2021)
- [Drifter: Efficient Online Feature Monitoring for Improved Data Integrity in Large-Scale Recommendation Systems](https://mlsanity.com/paper/2309.08617/) (2023)

## Abstract (arXiv)

Feature selection can be a crucial factor in obtaining robust and accurate predictions. Online feature selection models, however, operate under considerable restrictions; they need to efficiently extract salient input features based on a bounded set of observations, while enabling robust and accurate predictions. In this work, we introduce FIRES, a novel framework for online feature selection. The proposed feature weighting mechanism leverages the importance information inherent in the parameters of a predictive model. By treating model parameters as random variables, we can penalize features with high uncertainty and thus generate more stable feature sets. Our framework is generic in that it leaves the choice of the underlying model to the user. Strikingly, experiments suggest that the model complexity has only a minor effect on the discriminative power and stability of the selected feature sets. In fact, using a simple linear model, FIRES obtains feature sets that compete with state-of-the-art methods, while dramatically reducing computation time. In addition, experiments show that the proposed framework is clearly superior in terms of feature selection stability.