# Model-independent variable selection via the rule-based variable priority

arXiv: https://arxiv.org/abs/2409.09003 · HTML: https://mlsanity.com/paper/2409.09003/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 69 references that are in the map.
- First posted on arXiv 13 Sep 2024.

## Builds on (most cited first, 9 of 9)

- [Distribution-Free Predictive Inference For Regression](https://mlsanity.com/paper/1604.04173/) (2016)
- [All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously](https://mlsanity.com/paper/1801.01489/) (2018)
- [ranger: A Fast Implementation of Random Forests for High Dimensional Data in C++ and R](https://mlsanity.com/paper/1508.04409/) (2015)
- [Panning for Gold: Model-X Knockoffs for High-dimensional Controlled Variable Selection](https://mlsanity.com/paper/1610.02351/) (2016)
- [A general framework for inference on algorithm-agnostic variable importance](https://mlsanity.com/paper/2004.03683/) (2020)
- [Feature Selection with the R Package MXM: Discovering Statistically-Equivalent Feature Subsets](https://mlsanity.com/paper/1611.03227/) (2016)
- [Variable Selection with ABC Bayesian Forests](https://mlsanity.com/paper/1806.02304/) (2018)
- [COBRA: A Combined Regression Strategy](https://mlsanity.com/paper/1303.2236/) (2013)
- [A Model-free Variable Screening Method Based on Leverage Score](https://mlsanity.com/paper/2109.09936/) (2021)

## Built on by (most cited first, 2 of 2)

- [General OOD Detection via Model-aware and Subspace-aware Variable Priority](https://mlsanity.com/paper/2512.13003/) (2025)
- [Random Hazard Forests](https://mlsanity.com/paper/2608.21597/) (2026)

## Abstract (arXiv)

While achieving high prediction accuracy is a fundamental goal in machine learning, an equally important task is finding a small number of features with high explanatory power. One popular selection technique is permutation importance, which assesses a variable's impact by measuring the change in prediction error after permuting the variable. However, this can be problematic due to the need to create artificial data, a problem shared by other methods as well. Another problem is that variable selection methods can be limited by being model-specific. We introduce a new model-independent approach, Variable Priority (VarPro), which works by utilizing rules without the need to generate artificial data or evaluate prediction error. The method is relatively easy to use, requiring only the calculation of sample averages of simple statistics, and can be applied to many data settings, including regression, classification, and survival. We investigate the asymptotic properties of VarPro and show, among other things, that VarPro has a consistent filtering property for noise variables. Empirical studies using synthetic and real-world data show the method achieves a balanced performance and compares favorably to many state-of-the-art procedures currently used for variable selection.