# Tuning diagonal scale matrices for HMC

arXiv: https://arxiv.org/abs/2403.07495 · HTML: https://mlsanity.com/paper/2403.07495/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 53 references that are in the map.
- First posted on arXiv 12 Mar 2024.

## Builds on (most cited first, 13 of 13)

- [Rank-normalization, folding, and localization: An improved widehatR for assessing convergence of MCMC](https://mlsanity.com/paper/1903.08008/) (2019)
- [Geometric integrators and the Hamiltonian Monte Carlo method](https://mlsanity.com/paper/1711.05337/) (2017)
- [Piecewise Deterministic Markov Processes for Continuous-Time Monte Carlo](https://mlsanity.com/paper/1611.07873/) (2016)
- [Mixing Time Guarantees for Unadjusted Hamiltonian Monte Carlo](https://mlsanity.com/paper/2105.00887/) (2021)
- [When does Metropolized Hamiltonian Monte Carlo provably outperform Metropolis-adjusted Langevin algorithm?](https://mlsanity.com/paper/2304.04724/) (2023)
- [Leave Pima Indians alone: binary regression as a benchmark for Bayesian computation](https://mlsanity.com/paper/1506.08640/) (2015)
- [Connecting the Dots: Numerical Randomized Hamiltonian Monte Carlo with State-Dependent Event Rates](https://mlsanity.com/paper/2005.01285/) (2020)
- [Hamiltonian Monte Carlo for efficient Gaussian sampling: long and random steps](https://mlsanity.com/paper/2209.12771/) (2022)
- [Dynamically rescaled Hamiltonian Monte Carlo for Bayesian Hierarchical Models](https://mlsanity.com/paper/1806.02068/) (2018)
- [Randomized Time Riemannian Manifold Hamiltonian Monte Carlo](https://mlsanity.com/paper/2206.04554/) (2022)
- [Log-density gradient covariance and automatic metric tensors for Riemann manifold Monte Carlo methods](https://mlsanity.com/paper/2211.01746/) (2022)
- [Modified Cholesky Riemann Manifold Hamiltonian Monte Carlo: Exploiting Sparsity for Fast Sampling of High-dimensional Targets](https://mlsanity.com/paper/1612.04093/) (2016)
- [Couplings for Andersen Dynamics](https://mlsanity.com/paper/2009.14239/) (2020)

## Built on by (most cited first, 4 of 4)

- [Variational Inference for Uncertainty Quantification: an Analysis of Trade-offs](https://mlsanity.com/paper/2403.13748/) (2024)
- [The Within-Orbit Adaptive Leapfrog No-U-Turn Sampler](https://mlsanity.com/paper/2506.18746/) (2025)
- [Numerical Generalized Randomized Hamiltonian Monte Carlo for piecewise smooth target densities](https://mlsanity.com/paper/2504.18210/) (2025)
- [High-dimensional Adaptive MCMC with Reduced Computational Complexity](https://mlsanity.com/paper/2604.09286/) (2026)

## Abstract (arXiv)

Three approaches for adaptively tuning diagonal scale matrices for HMC are discussed and compared. The common practice of scaling according to estimated marginal standard deviations is taken as a benchmark. Scaling according to the mean log-target gradient (ISG), and a scaling method targeting that the frequency of when the underlying Hamiltonian dynamics crosses the respective medians should be uniform across dimensions, are taken as alternatives. Numerical studies suggest that the ISG method leads in many cases to more efficient sampling than the benchmark, in particular in cases with strong correlations or non-linear dependencies. The ISG method is also easy to implement, computationally cheap and would be relatively simple to include in automatically tuned codes as an alternative to the benchmark practice.