# Conjugate gradient methods for high-dimensional GLMMs

arXiv: https://arxiv.org/abs/2411.04729 · HTML: https://mlsanity.com/paper/2411.04729/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 49 references that are in the map.
- First posted on arXiv 7 Nov 2024.

## Builds on (most cited first, 12 of 12)

- [A general framework for Vecchia approximations of Gaussian processes](https://mlsanity.com/paper/1708.06302/) (2017)
- [Sparse Cholesky factorization by Kullback-Leibler minimization](https://mlsanity.com/paper/2004.14455/) (2020)
- [Gaussian variational approximation with sparse precision matrices](https://mlsanity.com/paper/1605.05622/) (2016)
- [Partially factorized variational inference for high-dimensional mixed models](https://mlsanity.com/paper/2312.13148/) (2023)
- [Scalable Bayesian computation for crossed and nested hierarchical models](https://mlsanity.com/paper/2103.10875/) (2021)
- [Scalable inference for crossed random effects models](https://mlsanity.com/paper/1803.09460/) (2018)
- [Prior-preconditioned conjugate gradient method for accelerated Gibbs sampling in "large n & large p" Bayesian sparse regression](https://mlsanity.com/paper/1810.12437/) (2018)
- [Backfitting for large scale crossed random effects regressions](https://mlsanity.com/paper/2007.10612/) (2020)
- [Efficient moment calculations for variance components in large unbalanced crossed random effects models](https://mlsanity.com/paper/1602.00346/) (2016)
- [Streamlined Variational Inference for Linear Mixed Models with Crossed Random Effects](https://mlsanity.com/paper/1910.01799/) (2019)
- [Estimation and Inference for Very Large Linear Mixed Effects Models](https://mlsanity.com/paper/1610.08088/) (2016)
- [Linear-cost unbiased posterior estimates for crossed effects and matrix factorization models via couplings](https://mlsanity.com/paper/2410.08939/) (2024)

## Built on by (most cited first, 2 of 2)

- [Mixing times of data-augmentation Gibbs samplers for high-dimensional probit regression](https://mlsanity.com/paper/2505.14343/) (2025)
- [Scalable Krylov Subspace Methods for Generalized Mixed-Effects Models with Crossed Random Effects](https://mlsanity.com/paper/2505.09552/) (2025)

## Abstract (arXiv)

Generalized linear mixed models (GLMMs) are a widely used tool in statistical analysis. The main bottleneck of many computational approaches lies in the inversion of the high dimensional precision matrices associated with the random effects. Such matrices are typically sparse; however, the sparsity pattern resembles a multi partite random graph, which does not lend itself well to default sparse linear algebra techniques. Notably, we show that, for typical GLMMs, the Cholesky factor is dense even when the original precision is sparse. We thus turn to approximate iterative techniques, in particular to the conjugate gradient (CG) method. We combine a detailed analysis of the spectrum of said precision matrices with results from random graph theory to show that CG-based methods applied to high-dimensional GLMMs typically achieve a fixed approximation error with a total cost that scales linearly with the number of parameters and observations. Numerical illustrations with both real and simulated data confirm the theoretical findings, while at the same time illustrating situations, such as nested structures, where CG-based methods struggle.