# A Consistent and Scalable Algorithm for Best Subset Selection in Single Index Models

arXiv: https://arxiv.org/abs/2309.06230 · HTML: https://mlsanity.com/paper/2309.06230/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 55 references that are in the map.
- First posted on arXiv 12 Sep 2023.

## Builds on (most cited first, 11 of 11)

- [Loss minimization and parameter estimation with heavy tails](https://mlsanity.com/paper/1307.1827/) (2013)
- [Sparse Regression: Scalable algorithms and empirical performance](https://mlsanity.com/paper/1902.06547/) (2019)
- [abess: A Fast Best Subset Selection Library in Python and R](https://mlsanity.com/paper/2110.09697/) (2021)
- [Tuning parameter selection in high dimensional penalized likelihood](https://mlsanity.com/paper/1605.03321/) (2016)
- [Understanding Implicit Regularization in Over-Parameterized Single Index Model](https://mlsanity.com/paper/2007.08322/) (2020)
- [L1-Regularized Least Squares for Support Recovery of High Dimensional Single Index Models with Gaussian Designs](https://mlsanity.com/paper/1511.08102/) (2015)
- [On Learning High Dimensional Structured Single Index Models](https://mlsanity.com/paper/1603.03980/) (2016)
- [High-dimensional Varying Index Coefficient Models via Stein's Identity](https://mlsanity.com/paper/1810.07128/) (2018)
- [Rank-based Lasso -- efficient methods for high-dimensional robust model selection](https://mlsanity.com/paper/1905.05876/) (2019)
- [A Splicing Approach to Best Subset of Groups Selection](https://mlsanity.com/paper/2104.12576/) (2021)
- [Best-Subset Selection in Generalized Linear Models: A Fast and Consistent Algorithm via Splicing Technique](https://mlsanity.com/paper/2308.00251/) (2023)

## Abstract (arXiv)

Analysis of high-dimensional data has led to increased interest in both single index models (SIMs) and the best-subset selection. SIMs provide an interpretable and flexible modeling framework for high-dimensional data, while the best-subset selection aims to find a sparse model from a large set of predictors. However, the best-subset selection in high-dimensional models is known to be computationally intractable. Existing proxy algorithms are appealing but do not yield the bestsubset solution. In this paper, we directly tackle the intractability by proposing a provably scalable algorithm for the best-subset selection in high-dimensional SIMs. We directly proved the subset selection consistency and oracle property for our algorithmic solution, distinguishing it from other state-of-the-art support recovery methods in SIMs. The algorithm comprises a generalized information criterion to determine the support size of the regression coefficients, eliminating the model selection tuning. Moreover, our method does not assume an error distribution or a specific link function and hence is flexible to apply. Extensive simulation results demonstrate that our method is not only computationally efficient but also able to exactly recover the best subset in various settings (e.g., linear regression, Poisson regression, heteroscedastic models).