# PPFS: Predictive Permutation Feature Selection

arXiv: https://arxiv.org/abs/2110.10713 · HTML: https://mlsanity.com/paper/2110.10713/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 67 references that are in the map.
- First posted on arXiv 20 Oct 2021.

## Builds on (most cited first, 8 of 8)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Deep Domain Confusion: Maximizing for Domain Invariance](https://mlsanity.com/paper/1412.3474/) (2014)
- [Panning for Gold: Model-X Knockoffs for High-dimensional Controlled Variable Selection](https://mlsanity.com/paper/1610.02351/) (2016)
- [Controlling the false discovery rate via knockoffs](https://mlsanity.com/paper/1404.5609/) (2014)
- [Approximate Kernel-based Conditional Independence Tests for Fast Non-Parametric Causal Discovery](https://mlsanity.com/paper/1702.03877/) (2017)
- [Model-Powered Conditional Independence Test](https://mlsanity.com/paper/1709.06138/) (2017)
- [Testing Conditional Independence in Supervised Learning Algorithms](https://mlsanity.com/paper/1901.09917/) (2019)

## Built on by (most cited first, 2 of 2)

- [Beyond Chemical Language: A Multimodal Approach to Enhance Molecular Property Prediction](https://mlsanity.com/paper/2306.14919/) (2023)
- [Attribute Graphs Underlying Molecular Generative Models: Path to Learning with Limited Data](https://mlsanity.com/paper/2207.07174/) (2022)

## Abstract (arXiv)

We propose Predictive Permutation Feature Selection (PPFS), a novel wrapper-based feature selection method based on the concept of Markov Blanket (MB). Unlike previous MB methods, PPFS is a universal feature selection technique as it can work for both classification as well as regression tasks on datasets containing categorical and/or continuous features. We propose Predictive Permutation Independence (PPI), a new Conditional Independence (CI) test, which enables PPFS to be categorised as a wrapper feature selection method. This is in contrast to current filter based MB feature selection techniques that are unable to harness the advancements in supervised algorithms such as Gradient Boosting Machines (GBM). The PPI test is based on the knockoff framework and utilizes supervised algorithms to measure the association between an individual or a set of features and the target variable. We also propose a novel MB aggregation step that addresses the issue of sample inefficiency. Empirical evaluations and comparisons on a large number of datasets demonstrate that PPFS outperforms state-of-the-art Markov blanket discovery algorithms as well as, well-known wrapper methods. We also provide a sketch of the proof of correctness of our method. Implementation of this work is available at urlhttps://github.com/atif-hassan/PyImpetus