# Fast Riemannian-manifold Hamiltonian Monte Carlo for hierarchical Gaussian-process models

arXiv: https://arxiv.org/abs/2511.06407 · HTML: https://mlsanity.com/paper/2511.06407/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 60 references that are in the map.
- First posted on arXiv 9 Nov 2025.

## Builds on (most cited first, 15 of 15)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [GPflow: A Gaussian process library using TensorFlow](https://mlsanity.com/paper/1610.08733/) (2016)
- [Underdamped Langevin MCMC: A non-asymptotic analysis](https://mlsanity.com/paper/1707.03663/) (2017)
- [Hilbert Space Methods for Reduced-Rank Gaussian Process Regression](https://mlsanity.com/paper/1401.5508/) (2014)
- [On sampling from a log-concave density using kinetic Langevin diffusions](https://mlsanity.com/paper/1807.09382/) (2018)
- [MCMC for Variationally Sparse Gaussian Processes](https://mlsanity.com/paper/1506.04000/) (2015)
- [Bayesian Inference and Learning in Gaussian Process State-Space Models with Particle MCMC](https://mlsanity.com/paper/1306.2861/) (2013)
- [A Bayesian information criterion for singular models](https://mlsanity.com/paper/1309.0911/) (2013)
- [Computationally Efficient Bayesian Learning of Gaussian Process State Space Models](https://mlsanity.com/paper/1506.02267/) (2015)
- [Bayesian Computing with INLA: A Review](https://mlsanity.com/paper/1604.00860/) (2016)
- [Improved Rectangular Matrix Multiplication using Powers of the Coppersmith-Winograd Tensor](https://mlsanity.com/paper/1708.05622/) (2017)
- [Introducing an Explicit Symplectic Integration Scheme for Riemannian Manifold Hamiltonian Monte Carlo](https://mlsanity.com/paper/1910.06243/) (2019)
- [Generalizing the No-U-Turn Sampler to Riemannian Manifolds](https://mlsanity.com/paper/1304.1920/) (2013)
- [Fully Bayesian inference for latent variable Gaussian process models](https://mlsanity.com/paper/2211.02218/) (2022)
- [Debiased maximum-likelihood estimators for hazard ratios under kernel-based machine-learning adjustment](https://mlsanity.com/paper/2507.17686/) (2025)

## Abstract (arXiv)

Hierarchical Bayesian models based on Gaussian processes are considered useful for describing complex nonlinear statistical dependencies among variables in real-world data. However, effective Monte Carlo algorithms for inference with these models have not yet been established, except for several simple cases. In this study, we show that, compared with the slow inference achieved with existing program libraries, the performance of Riemannian-manifold Hamiltonian Monte Carlo (RMHMC) can be drastically improved by optimising the computation order according to the model structure and dynamically programming the eigendecomposition. This improvement cannot be achieved when using an existing library based on a naive automatic differentiator. We numerically demonstrate that RMHMC effectively samples from the posterior, allowing the calculation of model evidence, in a Bayesian logistic regression on simulated data and in the estimation of propensity functions for the American national medical expenditure data using several Bayesian multiple-kernel models. These results lay a foundation for implementing effective Monte Carlo algorithms for analysing real-world data with Gaussian processes, and highlight the need to develop a customisable library set that allows users to incorporate dynamically programmed objects and finely optimises the mode of automatic differentiation depending on the model structure.