# Finding Optimal Diverse Feature Sets with Alternative Feature Selection

arXiv: https://arxiv.org/abs/2307.11607 · HTML: https://mlsanity.com/paper/2307.11607/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 87 references that are in the map.
- First posted on arXiv 21 Jul 2023.

## Builds on (most cited first, 15 of 15)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017)
- [Attention is not Explanation](https://mlsanity.com/paper/1902.10186/) (2019)
- [Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations](https://mlsanity.com/paper/1905.07697/) (2019)
- [Model-Agnostic Counterfactual Explanations for Consequential Decisions](https://mlsanity.com/paper/1905.11190/) (2019)
- [PMLB: A Large Benchmark Suite for Machine Learning Evaluation and Comparison](https://mlsanity.com/paper/1703.00512/) (2017)
- [Interpretable Machine Learning -- A Brief History, State-of-the-Art and Challenges](https://mlsanity.com/paper/2010.09337/) (2020)
- [Efficient Search for Diverse Coherent Explanations](https://mlsanity.com/paper/1901.04909/) (2019)
- [Feature Selection: A Data Perspective](https://mlsanity.com/paper/1601.07996/) (2016)
- [Scaling Guarantees for Nearest Counterfactual Explanations](https://mlsanity.com/paper/2010.04965/) (2020)
- [A multi-component framework for the analysis and design of explainable artificial intelligence](https://mlsanity.com/paper/2005.01908/) (2020)
- ["Even if ..." -- Diverse Semifactual Explanations of Reject](https://mlsanity.com/paper/2207.01898/) (2022)
- [Feature Selection with the R Package MXM: Discovering Statistically-Equivalent Feature Subsets](https://mlsanity.com/paper/1611.03227/) (2016)
- [Efficient Subspace Search in Data Streams](https://mlsanity.com/paper/2011.06959/) (2020)

## Built on by (most cited first, 1 of 1)

- [GEMSS: A Variational Method for Discovering Multiple Sparse Solutions in Classification and Regression Problems](https://mlsanity.com/paper/2602.08913/) (2026)

## Abstract (arXiv)

Feature selection is popular for obtaining small, interpretable, yet highly accurate prediction models. Conventional feature-selection methods typically yield one feature set only, which might not suffice in some scenarios. For example, users might be interested in finding alternative feature sets with similar prediction quality, offering different explanations of the data. In this article, we introduce alternative feature selection and formalize it as an optimization problem. In particular, we define alternatives via constraints and enable users to control the number and dissimilarity of alternatives. We consider sequential as well as simultaneous search for alternatives. Next, we discuss how to integrate conventional feature-selection methods as objectives. In particular, we describe solver-based search methods to tackle the optimization problem. Further, we analyze the complexity of this optimization problem and prove NP-hardness. Additionally, we show that a constant-factor approximation exists under certain conditions and propose corresponding heuristic search methods. Finally, we evaluate alternative feature selection in comprehensive experiments with 30 binary-classification datasets. We observe that alternative feature sets may indeed have high prediction quality, and we analyze factors influencing this outcome.