# Subset Selection for Multiple Linear Regression via Optimization

arXiv: https://arxiv.org/abs/1701.07920 · HTML: https://mlsanity.com/paper/1701.07920/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Jan 2017.

## Built on by (most cited first, 5 of 5)

- [Bilevel Optimization for Feature Selection in the Data-Driven Newsvendor Problem](https://mlsanity.com/paper/2209.05093/) (2022)
- [Outlier detection in regression: conic quadratic formulations](https://mlsanity.com/paper/2307.05975/) (2023)
- [Scalable First-order Method for Certifying Optimal k-Sparse GLMs](https://mlsanity.com/paper/2502.09502/) (2025)
- [Integrated Subset Selection and Bandwidth Estimation Algorithm for Geographically Weighted Regression](https://mlsanity.com/paper/2503.17253/) (2025)
- [Buffered AUC maximization for scoring systems via mixed-integer optimization](https://mlsanity.com/paper/2601.05544/) (2026)

## Abstract (arXiv)

Subset selection in multiple linear regression aims to choose a subset of candidate explanatory variables that tradeoff fitting error (explanatory power) and model complexity (number of variables selected). We build mathematical programming models for regression subset selection based on mean square and absolute errors, and minimal-redundancy-maximal-relevance criteria. The proposed models are tested using a linear-program-based branch-and-bound algorithm with tailored valid inequalities and big M values and are compared against the algorithms in the literature. For high dimensional cases, an iterative heuristic algorithm is proposed based on the mathematical programming models and a core set concept, and a randomized version of the algorithm is derived to guarantee convergence to the global optimum. From the computational experiments, we find that our models quickly find a quality solution while the rest of the time is spent to prove optimality; the iterative algorithms find solutions in a relatively short time and are competitive compared to state-of-the-art algorithms; using ad-hoc big M values is not recommended.