# Repelling-Attracting Hamiltonian Monte Carlo

arXiv: https://arxiv.org/abs/2403.04607 · HTML: https://mlsanity.com/paper/2403.04607/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 73 references that are in the map.
- First posted on arXiv 7 Mar 2024.

## Builds on (most cited first, 24 of 24)

- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Stochastic Gradient Hamiltonian Monte Carlo](https://mlsanity.com/paper/1402.4102/) (2014)
- [A Conceptual Introduction to Hamiltonian Monte Carlo](https://mlsanity.com/paper/1701.02434/) (2017)
- [Statistical Aspects of Wasserstein Distances](https://mlsanity.com/paper/1806.05500/) (2018)
- [Generalizing Hamiltonian Monte Carlo with Neural Networks](https://mlsanity.com/paper/1711.09268/) (2017)
- [Conformal Symplectic and Relativistic Optimization](https://mlsanity.com/paper/1903.04100/) (2019)
- [Optimization with Momentum: Dynamical, Control-Theoretic, and Symplectic Perspectives](https://mlsanity.com/paper/2002.12493/) (2020)
- [A Framework for Adaptive MCMC Targeting Multimodal Distributions](https://mlsanity.com/paper/1812.02609/) (2018)
- [The Geometric Foundations of Hamiltonian Monte Carlo](https://mlsanity.com/paper/1410.5110/) (2014)
- [On the Geometric Ergodicity of Hamiltonian Monte Carlo](https://mlsanity.com/paper/1601.08057/) (2016)
- [Gradient-free Hamiltonian Monte Carlo with Efficient Kernel Exponential Families](https://mlsanity.com/paper/1506.02564/) (2015)
- [Pseudo-Marginal Hamiltonian Monte Carlo](https://mlsanity.com/paper/1607.02516/) (2016)
- [Thermostat-assisted continuously-tempered Hamiltonian Monte Carlo for Bayesian learning](https://mlsanity.com/paper/1711.11511/) (2017)
- [Relativistic Monte Carlo](https://mlsanity.com/paper/1609.04388/) (2016)
- [Magnetic Hamiltonian Monte Carlo](https://mlsanity.com/paper/1607.02738/) (2016)
- [Quantum-Inspired Hamiltonian Monte Carlo for Bayesian Sampling](https://mlsanity.com/paper/1912.01937/) (2019)
- [Geometry and Dynamics for Markov Chain Monte Carlo](https://mlsanity.com/paper/1705.02891/) (2017)
- [Pseudo-extended Markov chain Monte Carlo](https://mlsanity.com/paper/1708.05239/) (2017)
- [Adiabatic Monte Carlo](https://mlsanity.com/paper/1405.3489/) (2014)
- [A Repelling-Attracting Metropolis Algorithm for Multimodality](https://mlsanity.com/paper/1601.05633/) (2016)
- [Does Hamiltonian Monte Carlo mix faster than a random walk on multimodal densities?](https://mlsanity.com/paper/1808.03230/) (2018)
- [Non-Canonical Hamiltonian Monte Carlo](https://mlsanity.com/paper/2008.08191/) (2020)
- [Globally-centered autocovariances in MCMC](https://mlsanity.com/paper/2009.01799/) (2020)
- [Sampling from high-dimensional, multimodal distributions using automatically tuned, tempered Hamiltonian Monte Carlo](https://mlsanity.com/paper/2111.06871/) (2021)

## Abstract (arXiv)

We propose a variant of Hamiltonian Monte Carlo (HMC), called the Repelling-Attracting Hamiltonian Monte Carlo (RAHMC), for sampling from multimodal distributions. The key idea that underpins RAHMC is a departure from the conservative dynamics of Hamiltonian systems, which form the basis of traditional HMC, and turning instead to the dissipative dynamics of conformal Hamiltonian systems. In particular, RAHMC involves two stages: a mode-repelling stage to encourage the sampler to move away from regions of high probability density; and, a mode-attracting stage, which facilitates the sampler to find and settle near alternative modes. We achieve this by introducing just one additional tuning parameter -- the coefficient of friction. The proposed method adapts to the geometry of the target distribution, e.g., modes and density ridges, and can generate proposals that cross low-probability barriers with little to no computational overhead in comparison to traditional HMC. Notably, RAHMC requires no additional information about the target distribution or memory of previously visited modes. We establish the theoretical basis for RAHMC, and we discuss repelling-attracting extensions to several variants of HMC in literature. Finally, we provide a tuning-free implementation via dual-averaging, and we demonstrate its effectiveness in sampling from, both, multimodal and unimodal distributions in high dimensions.