Optimal scaling of MCMC algorithms: the Hamiltonian approach

We present a simple, yet general approach to study the scaling properties as the dimensionality of Metropolised MCMC sampling algorithms increases. The study relies on the symmetries of the Hamiltonian formalism and ultimately on the symmetry of the Metropolis-Hastings formula. Our findings contain, as particular cases, many known results for the Random Walk Metropolis, MALA and other algorithms. In addition, they provide, in an easy way, new optimal scaling results for a variety of proposal mechanisms, including implicit proposals and proposals generated with the help of differential equation integrators. The analysis applies to targets that are products of a given, not necessarily univariate distribution, and also to cases where the different terms in the product are scaled differently. We show how to construct gradient-based MALA-like proposals where the variance of the proposal as the dimension d increases may be taken as O(1/dμ), with μ>0 arbitrarily small, to be compared with the values μ = 1 for Random Walk Metropolis and μ=1/3 for MALA.