# Feature subset selection for kernel SVM classification via mixed-integer optimization

arXiv: https://arxiv.org/abs/2205.14325 · HTML: https://mlsanity.com/paper/2205.14325/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 77 references that are in the map.
- First posted on arXiv 28 May 2022.

## Builds on (most cited first, 12 of 12)

- [Best Subset Selection via a Modern Optimization Lens](https://mlsanity.com/paper/1507.03133/) (2015)
- [Supersparse Linear Integer Models for Optimized Medical Scoring Systems](https://mlsanity.com/paper/1502.04269/) (2015)
- [Fast Best Subset Selection: Coordinate Descent and Local Combinatorial Optimization Algorithms](https://mlsanity.com/paper/1803.01454/) (2018)
- [Sparse Regression at Scale: Branch-and-Bound rooted in First-Order Optimization](https://mlsanity.com/paper/2004.06152/) (2020)
- [Sparse Regression: Scalable algorithms and empirical performance](https://mlsanity.com/paper/1902.06547/) (2019)
- [Learning Sparse Classifiers: Continuous and Mixed Integer Optimization Perspectives](https://mlsanity.com/paper/2001.06471/) (2020)
- [Sparse Classification: a scalable discrete optimization perspective](https://mlsanity.com/paper/1710.01352/) (2017)
- [A novel embedded min-max approach for feature selection in nonlinear support vector machine classification](https://mlsanity.com/paper/2004.09863/) (2020)
- [How to tune the RBF SVM hyperparameters?: An empirical evaluation of 18 search algorithms](https://mlsanity.com/paper/2008.11655/) (2020)
- [Scalable Holistic Linear Regression](https://mlsanity.com/paper/1902.03272/) (2019)
- [Mixed Integer Linear Programming for Feature Selection in Support Vector Machine](https://mlsanity.com/paper/1808.02435/) (2018)
- [Piecewise-Linear Approximation for Feature Subset Selection in a Sequential Logit Model](https://mlsanity.com/paper/1510.05417/) (2015)

## Built on by (most cited first, 1 of 1)

- [Cost-Sensitive Best Subset Selection for Logistic Regression: A Mixed-Integer Conic Optimization Perspective](https://mlsanity.com/paper/2310.05464/) (2023)

## Abstract (arXiv)

We study the mixed-integer optimization (MIO) approach to feature subset selection in nonlinear kernel support vector machines (SVMs) for binary classification. First proposed for linear regression in the 1970s, this approach has recently moved into the spotlight with advances in optimization algorithms and computer hardware. The goal of this paper is to establish an MIO approach for selecting the best subset of features for kernel SVM classification. To measure the performance of subset selection, we use the kernel-target alignment, which is the distance between the centroids of two response classes in a high-dimensional feature space. We propose a mixed-integer linear optimization (MILO) formulation based on the kernel-target alignment for feature subset selection, and this MILO problem can be solved to optimality using optimization software. We also derive a reduced version of the MILO problem to accelerate our MILO computations. Experimental results show good computational efficiency for our MILO formulation with the reduced problem. Moreover, our method can often outperform the linear-SVM-based MILO formulation and recursive feature elimination in prediction performance, especially when there are relatively few data instances.