# Hamiltonian Monte Carlo Inference of Marginalized Linear Mixed-Effects Models

arXiv: https://arxiv.org/abs/2410.24079 · HTML: https://mlsanity.com/paper/2410.24079/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 76 references that are in the map.
- First posted on arXiv 31 Oct 2024.

## Builds on (most cited first, 27 of 27)

- [GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration](https://mlsanity.com/paper/1809.11165/) (2018)
- [Pyro: Deep Universal Probabilistic Programming](https://mlsanity.com/paper/1810.09538/) (2018)
- [Composable Effects for Flexible and Accelerated Probabilistic Programming in NumPyro](https://mlsanity.com/paper/1912.11554/) (2019)
- [TensorFlow Distributions](https://mlsanity.com/paper/1711.10604/) (2017)
- [NeuTra-lizing Bad Geometry in Hamiltonian Monte Carlo Using Neural Transport](https://mlsanity.com/paper/1903.03704/) (2019)
- [Yes, but Did It Work?: Evaluating Variational Inference](https://mlsanity.com/paper/1802.02538/) (2018)
- [Rao-Blackwellised Particle Filtering for Dynamic Bayesian Networks](https://mlsanity.com/paper/1301.3853/) (2013)
- [Integrating Random Effects in Deep Neural Networks](https://mlsanity.com/paper/2206.03314/) (2022)
- [Automatic Reparameterisation of Probabilistic Programs](https://mlsanity.com/paper/1906.03028/) (2019)
- [Lecture notes on ridge regression](https://mlsanity.com/paper/1509.09169/) (2015)
- [Deterministic Langevin Monte Carlo with Normalizing Flows for Bayesian Inference](https://mlsanity.com/paper/2205.14240/) (2022)
- [Scalable Bayesian computation for crossed and nested hierarchical models](https://mlsanity.com/paper/2103.10875/) (2021)
- [Hamiltonian Dynamics with Non-Newtonian Momentum for Rapid Sampling](https://mlsanity.com/paper/2111.02434/) (2021)
- [Delayed Sampling and Automatic Rao-Blackwellization of Probabilistic Programs](https://mlsanity.com/paper/1708.07787/) (2017)
- [Scalable inference for crossed random effects models](https://mlsanity.com/paper/1803.09460/) (2018)
- [Auto-Differentiating Linear Algebra](https://mlsanity.com/paper/1710.08717/) (2017)
- [Accelerating Hamiltonian Monte Carlo via Chebyshev Integration Time](https://mlsanity.com/paper/2207.02189/) (2022)
- [CoLA: Exploiting Compositional Structure for Automatic and Efficient Numerical Linear Algebra](https://mlsanity.com/paper/2309.03060/) (2023)
- [Neural Mixed Effects for Nonlinear Personalized Predictions](https://mlsanity.com/paper/2306.08149/) (2023)
- [Conditional independence by typing](https://mlsanity.com/paper/2010.11887/) (2020)
- [Semi-Symbolic Inference for Efficient Streaming Probabilistic Programming](https://mlsanity.com/paper/2209.07490/) (2022)
- [Automatically Marginalized MCMC in Probabilistic Programming](https://mlsanity.com/paper/2302.00564/) (2023)
- [Estimation and Inference for Very Large Linear Mixed Effects Models](https://mlsanity.com/paper/1610.08088/) (2016)
- [Bayesian Multinomial Logistic Normal Models through Marginally Latent Matrix-T Processes](https://mlsanity.com/paper/1903.11695/) (2019)
- [Scalable solution to crossed random effects model with random slopes](https://mlsanity.com/paper/2307.12378/) (2023)
- [Sparse Probit Linear Mixed Model](https://mlsanity.com/paper/1507.04777/) (2015)
- [Fast methods for posterior inference of two-group normal-normal models](https://mlsanity.com/paper/2110.03055/) (2021)

## Built on by (most cited first, 1 of 1)

- [Corrected Integrated Laplace Approximation for Bayesian Inference in Latent Gaussian Models](https://mlsanity.com/paper/2605.20345/) (2026)

## Abstract (arXiv)

Bayesian reasoning in linear mixed-effects models (LMMs) is challenging and often requires advanced sampling techniques like Markov chain Monte Carlo (MCMC). A common approach is to write the model in a probabilistic programming language and then sample via Hamiltonian Monte Carlo (HMC). However, there are many ways a user can transform a model that make inference more or less efficient. In particular, marginalizing some variables can greatly improve inference but is difficult for users to do manually. We develop an algorithm to easily marginalize random effects in LMMs. A naive approach introduces cubic time operations within an inference algorithm like HMC, but we reduce the running time to linear using fast linear algebra techniques. We show that marginalization is always beneficial when applicable and highlight improvements in various models, especially ones from cognitive sciences.