# Scalable Krylov Subspace Methods for Generalized Mixed-Effects Models with Crossed Random Effects

arXiv: https://arxiv.org/abs/2505.09552 · HTML: https://mlsanity.com/paper/2505.09552/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 63 references that are in the map.
- First posted on arXiv 14 May 2025.

## Builds on (most cited first, 22 of 22)

- [GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration](https://mlsanity.com/paper/1809.11165/) (2018)
- [Constant-Time Predictive Distributions for Gaussian Processes](https://mlsanity.com/paper/1803.06058/) (2018)
- [Scalable Log Determinants for Gaussian Process Kernel Learning](https://mlsanity.com/paper/1711.03481/) (2017)
- [Gaussian Process Boosting](https://mlsanity.com/paper/2004.02653/) (2020)
- [Preconditioning for Scalable Gaussian Process Hyperparameter Optimization](https://mlsanity.com/paper/2107.00243/) (2021)
- [Integrating Random Effects in Deep Neural Networks](https://mlsanity.com/paper/2206.03314/) (2022)
- [Regularized target encoding outperforms traditional methods in supervised machine learning with high cardinality features](https://mlsanity.com/paper/2104.00629/) (2021)
- [Partially factorized variational inference for high-dimensional mixed models](https://mlsanity.com/paper/2312.13148/) (2023)
- [Scalable Bayesian computation for crossed and nested hierarchical models](https://mlsanity.com/paper/2103.10875/) (2021)
- [Latent Gaussian Model Boosting](https://mlsanity.com/paper/2105.08966/) (2021)
- [Scalable inference for crossed random effects models](https://mlsanity.com/paper/1803.09460/) (2018)
- [Machine Learning with High-Cardinality Categorical Features in Actuarial Applications](https://mlsanity.com/paper/2301.12710/) (2023)
- [Prior-preconditioned conjugate gradient method for accelerated Gibbs sampling in "large n & large p" Bayesian sparse regression](https://mlsanity.com/paper/1810.12437/) (2018)
- [Backfitting for large scale crossed random effects regressions](https://mlsanity.com/paper/2007.10612/) (2020)
- [Iterative Methods for Vecchia-Laplace Approximations for Latent Gaussian Process Models](https://mlsanity.com/paper/2310.12000/) (2023)
- [Efficient moment calculations for variance components in large unbalanced crossed random effects models](https://mlsanity.com/paper/1602.00346/) (2016)
- [Scalable logistic regression with crossed random effects](https://mlsanity.com/paper/2105.13747/) (2021)
- [Consistent and Scalable Composite Likelihood Estimation of Probit Models with Crossed Random Effects](https://mlsanity.com/paper/2308.15681/) (2023)
- [Iterative Methods for Full-Scale Gaussian Process Approximations for Large Spatial Data](https://mlsanity.com/paper/2405.14492/) (2024)
- [Estimation and Inference for Very Large Linear Mixed Effects Models](https://mlsanity.com/paper/1610.08088/) (2016)
- [A Comparison of Machine Learning Methods for Data with High-Cardinality Categorical Variables](https://mlsanity.com/paper/2307.02071/) (2023)
- [Conjugate gradient methods for high-dimensional GLMMs](https://mlsanity.com/paper/2411.04729/) (2024)

## Abstract (arXiv)

Mixed-effects models are widely used to model data with complex grouping structures and high-cardinality categorical predictor variables. However, for high-dimensional crossed random effects, current standard computations relying on Cholesky decompositions can become prohibitively slow. In this work, we present Krylov subspace-based methods that address existing computational bottlenecks, and we analyze them both theoretically and empirically. In particular, we derive new results on the convergence and accuracy of the preconditioned stochastic Lanczos quadrature and conjugate gradient methods for mixed-effects models, and we develop scalable methods for calculating predictive variances. In experiments with simulated and real-world data, the proposed methods yield speedups of several orders of magnitude and are more computationally robust than Cholesky-based computations, while maintaining essentially the same accuracy.