# Tuning parameter selection in high dimensional penalized likelihood

arXiv: https://arxiv.org/abs/1605.03321 · HTML: https://mlsanity.com/paper/1605.03321/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 May 2016.

## Built on by (most cited first, 13 of 13)

- [Best-Subset Selection in Generalized Linear Models: A Fast and Consistent Algorithm via Splicing Technique](https://mlsanity.com/paper/2308.00251/) (2023)
- [RaSE: A Variable Screening Framework via Random Subspace Ensembles](https://mlsanity.com/paper/2102.03892/) (2021)
- [Low-Rank Online Dynamic Assortment with Dual Contextual Information](https://mlsanity.com/paper/2404.17592/) (2024)
- [Statistical Jump Model for Mixed-Type Data with Missing Data Imputation](https://mlsanity.com/paper/2409.01208/) (2024)
- [Parallel integrative learning for large-scale multi-response regression with incomplete outcomes](https://mlsanity.com/paper/2104.05076/) (2021)
- [Two-Stage Robust and Sparse Distributed Statistical Inference for Large-Scale Data](https://mlsanity.com/paper/2208.08230/) (2022)
- [Pre-processing with Orthogonal Decompositions for High-dimensional Explanatory Variables](https://mlsanity.com/paper/2106.09071/) (2021)
- [Simultaneous Best Subset Selection and Dimension Reduction via Primal-Dual Iterations](https://mlsanity.com/paper/2211.15889/) (2022)
- [Gibbs-Based Information Criteria and the Over-Parameterized Regime](https://mlsanity.com/paper/2306.05583/) (2023)
- [A Consistent and Scalable Algorithm for Best Subset Selection in Single Index Models](https://mlsanity.com/paper/2309.06230/) (2023)
- [Distributed Linear Regression with Compositional Covariates](https://mlsanity.com/paper/2310.13969/) (2023)
- [Minimax and Communication-Efficient Distributed Best Subset Selection with Oracle Property](https://mlsanity.com/paper/2408.17276/) (2024)
- [Row-wise Fusion Regularization: An Interpretable Personalized Federated Learning Framework in Large-Scale Scenarios](https://mlsanity.com/paper/2510.14413/) (2025)

## Abstract (arXiv)

Determining how to appropriately select the tuning parameter is essential in penalized likelihood methods for high-dimensional data analysis. We examine this problem in the setting of penalized likelihood methods for generalized linear models, where the dimensionality of covariates p is allowed to increase exponentially with the sample size n. We propose to select the tuning parameter by optimizing the generalized information criterion (GIC) with an appropriate model complexity penalty. To ensure that we consistently identify the true model, a range for the model complexity penalty is identified in GIC. We find that this model complexity penalty should diverge at the rate of some power of log p depending on the tail probability behavior of the response variables. This reveals that using the AIC or BIC to select the tuning parameter may not be adequate for consistently identifying the true model. Based on our theoretical study, we propose a uniform choice of the model complexity penalty and show that the proposed approach consistently identifies the true model among candidate models with asymptotic probability one. We justify the performance of the proposed procedure by numerical simulations and a gene expression data analysis.