# High-dimensional classification by sparse logistic regression

arXiv: https://arxiv.org/abs/1706.08344 · HTML: https://mlsanity.com/paper/1706.08344/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jun 2017.

## Built on by (most cited first, 19 of 19)

- [Large Scale Prediction with Decision Trees](https://mlsanity.com/paper/2104.13881/) (2021)
- [Fast Rates for Noisy Interpolation Require Rethinking the Effects of Inductive Bias](https://mlsanity.com/paper/2203.03597/) (2022)
- [Leveraging Time Irreversibility with Order-Contrastive Pre-training](https://mlsanity.com/paper/2111.02599/) (2021)
- [Stability and Risk Bounds of Iterative Hard Thresholding](https://mlsanity.com/paper/2203.09413/) (2022)
- [Generalization Error Bounds for Multiclass Sparse Linear Classifiers](https://mlsanity.com/paper/2204.06264/) (2022)
- [Blessing of Class Diversity in Pre-training](https://mlsanity.com/paper/2209.03447/) (2022)
- [A Minimax Lower Bound for Low-Rank Matrix-Variate Logistic Regression](https://mlsanity.com/paper/2105.14673/) (2021)
- [Structured Low-Rank Tensors for Generalized Linear Models](https://mlsanity.com/paper/2308.02922/) (2023)
- [On Regularized Sparse Logistic Regression](https://mlsanity.com/paper/2309.05925/) (2023)
- [Statistical learning by sparse deep neural networks](https://mlsanity.com/paper/2311.08845/) (2023)
- [Adaptive posterior concentration rates for sparse high-dimensional linear regression with random design and unknown error variance](https://mlsanity.com/paper/2405.19016/) (2024)
- [Classification by sparse generalized additive models](https://mlsanity.com/paper/2212.01792/) (2022)
- [Tight bounds for maximum ℓ1-margin classifiers](https://mlsanity.com/paper/2212.03783/) (2022)
- [Globally Convergent Accelerated Algorithms for Multilinear Sparse Logistic Regression with ℓ0-constraints](https://mlsanity.com/paper/2309.09239/) (2023)
- [Misclassification excess risk bounds for 1-bit matrix completion](https://mlsanity.com/paper/2312.12945/) (2023)
- [Misclassification excess risk bounds for PAC-Bayesian classification via convexified loss](https://mlsanity.com/paper/2408.08675/) (2024)
- [High-dimensional prediction for count response via sparse exponential weights](https://mlsanity.com/paper/2410.15381/) (2024)
- [Kullback-Leibler excess risk bounds for exponential weighted aggregation in Generalized linear models](https://mlsanity.com/paper/2504.10171/) (2025)
- [Sparse classification with positive-confidence data in high dimensions](https://mlsanity.com/paper/2512.24443/) (2025)

## Abstract (arXiv)

We consider high-dimensional binary classification by sparse logistic regression. We propose a model/feature selection procedure based on penalized maximum likelihood with a complexity penalty on the model size and derive the non-asymptotic bounds for the resulting misclassification excess risk. The bounds can be reduced under the additional low-noise condition. The proposed complexity penalty is remarkably related to the VC-dimension of a set of sparse linear classifiers. Implementation of any complexity penalty-based criterion, however, requires a combinatorial search over all possible models. To find a model selection procedure computationally feasible for high-dimensional data, we extend the Slope estimator for logistic regression and show that under an additional weighted restricted eigenvalue condition it is rate-optimal in the minimax sense.