# Automatic Laplace Collapsed Sampling: Scalable Marginalisation of Latent Parameters via Automatic Differentiation

arXiv: https://arxiv.org/abs/2603.26644 · HTML: https://mlsanity.com/paper/2603.26644/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 56 references that are in the map.
- First posted on arXiv 27 Mar 2026.

## Builds on (most cited first, 17 of 17)

- [Efficient and Modular Implicit Differentiation](https://mlsanity.com/paper/2105.15183/) (2021)
- [Hamiltonian Monte Carlo for Hierarchical Models](https://mlsanity.com/paper/1312.0906/) (2013)
- [Student-t Processes as Alternatives to Gaussian Processes](https://mlsanity.com/paper/1402.4306/) (2014)
- [Marginal likelihood computation for model selection and hypothesis testing: an extensive review](https://mlsanity.com/paper/2005.08334/) (2020)
- [Automatic Reparameterisation of Probabilistic Programs](https://mlsanity.com/paper/1906.03028/) (2019)
- [Proximal nested sampling for high-dimensional Bayesian model selection](https://mlsanity.com/paper/2106.03646/) (2021)
- [Machine learning assisted Bayesian model comparison: learnt harmonic mean estimator](https://mlsanity.com/paper/2111.12720/) (2021)
- [Marginalised Gaussian Processes with Nested Sampling](https://mlsanity.com/paper/2010.16344/) (2020)
- [Automatically Marginalized MCMC in Probabilistic Programming](https://mlsanity.com/paper/2302.00564/) (2023)
- [Resonances in reflective Hamiltonian Monte Carlo](https://mlsanity.com/paper/2504.12374/) (2025)
- [Nested Slice Sampling: Vectorized Nested Sampling for GPU-Accelerated Inference](https://mlsanity.com/paper/2601.23252/) (2026)
- [MUSE: Marginal Unbiased Score Expansion and Application to CMB Lensing](https://mlsanity.com/paper/2112.09354/) (2021)
- [Marginal Post Processing of Bayesian Inference Products with Normalizing Flows and Kernel Density Estimators](https://mlsanity.com/paper/2205.12841/) (2022)
- [Is Gibbs sampling faster than Hamiltonian Monte Carlo on GLMs?](https://mlsanity.com/paper/2410.03630/) (2024)
- [Nested Sampling with Slice-within-Gibbs: Efficient Evidence Calculation for Hierarchical Bayesian Models](https://mlsanity.com/paper/2602.17414/) (2026)
- [aeons: approximating the end of nested sampling](https://mlsanity.com/paper/2312.00294/) (2023)
- [Fast Sampling of Cosmological Initial Conditions with Gaussian Neural Posterior Estimation](https://mlsanity.com/paper/2502.03139/) (2025)

## Abstract (arXiv)

We present Automatic Laplace Collapsed Sampling (ALCS), a general framework for marginalising latent parameters in Bayesian models using automatic differentiation, which we combine with nested sampling to explore the hyperparameter space in a robust and efficient manner. At each nested sampling likelihood evaluation, ALCS collapses the high-dimensional latent variables z to a scalar contribution via maximum a posteriori (MAP) optimisation and a Laplace approximation, both computed using autodiff. This reduces the effective dimension from dθ + dz to just dθ, making Bayesian evidence computation tractable for high-dimensional settings without hand-derived gradients or Hessians, and with minimal model-specific engineering. The MAP optimisation and Hessian evaluation are parallelised across live points on GPU-hardware, making the method practical at scale. We also show that automatic differentiation enables local approximations beyond Laplace to parametric families such as the Student-t, which improves evidence estimates for heavy-tailed latents. We validate ALCS on a suite of benchmarks spanning hierarchical, time-series, and discrete-likelihood models and establish where the Gaussian approximation holds. This enables a post-hoc ESS diagnostic that localises failures across hyperparameter space without expensive joint sampling.