# Subset selection for linear mixed models

arXiv: https://arxiv.org/abs/2107.12890 · HTML: https://mlsanity.com/paper/2107.12890/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 31 references that are in the map.
- First posted on arXiv 27 Jul 2021.

## Builds on (most cited first, 9 of 9)

- [Best Subset Selection via a Modern Optimization Lens](https://mlsanity.com/paper/1507.03133/) (2015)
- [Variable Importance Clouds: A Way to Explore Variable Importance for the Set of Good Models](https://mlsanity.com/paper/1901.03209/) (2019)
- [Decoupling shrinkage and selection in Bayesian linear models: a posterior summary perspective](https://mlsanity.com/paper/1408.0464/) (2014)
- [In Defense of the Indefensible: A Very Naive Approach to High-Dimensional Inference](https://mlsanity.com/paper/1705.05543/) (2017)
- [Prior-preconditioned conjugate gradient method for accelerated Gibbs sampling in "large n & large p" Bayesian sparse regression](https://mlsanity.com/paper/1810.12437/) (2018)
- [Bayesian subset selection and variable importance for interpretable prediction and classification](https://mlsanity.com/paper/2104.10150/) (2021)
- [Variable Selection in Seemingly Unrelated Regressions with Random Predictors](https://mlsanity.com/paper/1605.08963/) (2016)
- [Bayesian Function-on-Scalars Regression for High Dimensional Data](https://mlsanity.com/paper/1808.06689/) (2018)
- [Model Selection in Linear Mixed Models](https://mlsanity.com/paper/1306.2427/) (2013)

## Built on by (most cited first, 2 of 2)

- [Scalable Subset Selection in Linear Mixed Models](https://mlsanity.com/paper/2506.20425/) (2025)
- [Bayesian adaptive and interpretable functional regression for exposure profiles](https://mlsanity.com/paper/2203.00784/) (2022)

## Abstract (arXiv)

Linear mixed models (LMMs) are instrumental for regression analysis with structured dependence, such as grouped, clustered, or multilevel data. However, selection among the covariates--while accounting for this structured dependence--remains a challenge. We introduce a Bayesian decision analysis for subset selection with LMMs. Using a Mahalanobis loss function that incorporates the structured dependence, we derive optimal linear coefficients for (i) any given subset of variables and (ii) all subsets of variables that satisfy a cardinality constraint. Crucially, these estimates inherit shrinkage or regularization and uncertainty quantification from the underlying Bayesian model, and apply for any well-specified Bayesian LMM. More broadly, our decision analysis strategy deemphasizes the role of a single "best" subset, which is often unstable and limited in its information content, and instead favors a collection of near-optimal subsets. This collection is summarized by key member subsets and variable-specific importance metrics. Customized subset search and out-of-sample approximation algorithms are provided for more scalable computing. These tools are applied to simulated data and a longitudinal physical activity dataset, and demonstrate excellent prediction, estimation, and selection ability.