# Scalable Subset Selection in Linear Mixed Models

arXiv: https://arxiv.org/abs/2506.20425 · HTML: https://mlsanity.com/paper/2506.20425/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 49 references that are in the map.
- First posted on arXiv 25 Jun 2025.

## Builds on (most cited first, 12 of 12)

- [Coordinate Descent Algorithms](https://mlsanity.com/paper/1502.04759/) (2015)
- [Best Subset Selection via a Modern Optimization Lens](https://mlsanity.com/paper/1507.03133/) (2015)
- [Fast Best Subset Selection: Coordinate Descent and Local Combinatorial Optimization Algorithms](https://mlsanity.com/paper/1803.01454/) (2018)
- [Learning Sparse Classifiers: Continuous and Mixed Integer Optimization Perspectives](https://mlsanity.com/paper/2001.06471/) (2020)
- [Elastic Net Regularization Paths for All Generalized Linear Models](https://mlsanity.com/paper/2103.03475/) (2021)
- [Grouped Variable Selection with Discrete Optimization: Computational and Statistical Perspectives](https://mlsanity.com/paper/2104.07084/) (2021)
- [Non-asymptotic oracle inequalities for the Lasso in high-dimensional mixture of experts](https://mlsanity.com/paper/2009.10622/) (2020)
- [Bayesian subset selection and variable importance for interpretable prediction and classification](https://mlsanity.com/paper/2104.10150/) (2021)
- [A Relaxation Approach to Feature Selection for Linear Mixed Effects Models](https://mlsanity.com/paper/2205.06925/) (2022)
- [Subset selection for linear mixed models](https://mlsanity.com/paper/2107.12890/) (2021)
- [Group selection and shrinkage: Structured sparsity for semiparametric additive models](https://mlsanity.com/paper/2105.12081/) (2021)
- [glmmPen: High Dimensional Penalized Generalized Linear Mixed Models](https://mlsanity.com/paper/2305.08204/) (2023)

## Built on by (most cited first, 2 of 2)

- [Moment-Based Selection of Multiresponse Linear Mixed-Effects Models](https://mlsanity.com/paper/2607.01971/) (2026)
- [Structure Learning on Clustered Data](https://mlsanity.com/paper/2607.08238/) (2026)

## Abstract (arXiv)

Linear mixed models (LMMs), which incorporate fixed and random effects, are key tools for analyzing heterogeneous data, such as in personalized medicine. Nowadays, this type of data is increasingly wide, sometimes containing thousands of candidate predictors, necessitating sparsity for prediction and interpretation. However, existing sparse learning methods for LMMs do not scale well beyond tens or hundreds of predictors, leaving a large gap compared with sparse methods for linear models, which ignore random effects. This paper closes the gap with a new ℓ0 regularized method for LMM subset selection that can run on datasets containing thousands of predictors in seconds to minutes. On the computational front, we develop a coordinate descent algorithm as our main workhorse and provide a guarantee of its convergence. We also develop a local search algorithm to help traverse the nonconvex optimization surface. Both algorithms readily extend to subset selection in generalized LMMs via a penalized quasi-likelihood approximation. On the statistical front, we provide a finite-sample bound on the Kullback-Leibler divergence of the new method. We then demonstrate its excellent performance in experiments involving synthetic and real datasets.