# The Within-Orbit Adaptive Leapfrog No-U-Turn Sampler

arXiv: https://arxiv.org/abs/2506.18746 · HTML: https://mlsanity.com/paper/2506.18746/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 49 references that are in the map.
- First posted on arXiv 23 Jun 2025.

## Builds on (most cited first, 22 of 22)

- [A Conceptual Introduction to Hamiltonian Monte Carlo](https://mlsanity.com/paper/1701.02434/) (2017)
- [Hamiltonian Monte Carlo for Hierarchical Models](https://mlsanity.com/paper/1312.0906/) (2013)
- [Geometric integrators and the Hamiltonian Monte Carlo method](https://mlsanity.com/paper/1711.05337/) (2017)
- [The Shrinkage-Delinkage Trade-off: An Analysis of Factorized Gaussian Approximations for Variational Inference](https://mlsanity.com/paper/2302.09163/) (2023)
- [A general perspective on the Metropolis-Hastings kernel](https://mlsanity.com/paper/2012.14881/) (2020)
- [GIST: Gibbs self-tuning for locally adaptive Hamiltonian Monte Carlo](https://mlsanity.com/paper/2404.15253/) (2024)
- [Variational Inference in Location-Scale Families: Exact Recovery of the Mean and Correlation Matrix](https://mlsanity.com/paper/2410.11067/) (2024)
- [Connecting the Dots: Numerical Randomized Hamiltonian Monte Carlo with State-Dependent Event Rates](https://mlsanity.com/paper/2005.01285/) (2020)
- [Delayed rejection Hamiltonian Monte Carlo for sampling multiscale distributions](https://mlsanity.com/paper/2110.00610/) (2021)
- [Incorporating Local Step-Size Adaptivity into the No-U-Turn Sampler using Gibbs Self Tuning](https://mlsanity.com/paper/2408.08259/) (2024)
- [Couplings for Multinomial Hamiltonian Monte Carlo](https://mlsanity.com/paper/2104.05134/) (2021)
- [The Apogee to Apogee Path Sampler](https://mlsanity.com/paper/2112.08187/) (2021)
- [Mixing of the No-U-Turn Sampler and the Geometry of Gaussian Concentration](https://mlsanity.com/paper/2410.06978/) (2024)
- [Adaptive step size selection for Hessian-based manifold Langevin samplers](https://mlsanity.com/paper/1501.07454/) (2015)
- [Tuning diagonal scale matrices for HMC](https://mlsanity.com/paper/2403.07495/) (2024)
- [AutoStep: Locally adaptive involutive MCMC](https://mlsanity.com/paper/2410.18929/) (2024)
- [New affine invariant ensemble samplers and their dimensional scaling](https://mlsanity.com/paper/2505.02987/) (2025)
- [Dynamically rescaled Hamiltonian Monte Carlo for Bayesian Hierarchical Models](https://mlsanity.com/paper/1806.02068/) (2018)
- [Randomized Time Riemannian Manifold Hamiltonian Monte Carlo](https://mlsanity.com/paper/2206.04554/) (2022)
- [On the accept-reject mechanism for Metropolis-Hastings algorithms](https://mlsanity.com/paper/2011.04493/) (2020)
- [Towards practical PDMP sampling: Metropolis adjustments, locally adaptive step-sizes, and NUTS-based time lengths](https://mlsanity.com/paper/2503.11479/) (2025)
- [Hamiltonian Descent Algorithms for Optimization: Accelerated Rates via Randomized Integration Time](https://mlsanity.com/paper/2505.12553/) (2025)

## Built on by (most cited first, 2 of 2)

- [On Accelerated Mixing of the No-U-turn Sampler](https://mlsanity.com/paper/2507.13259/) (2025)
- [A Theoretical Comparison of No-U-Turn Sampler Variants: Necessary and Sufficient Convergence Conditions and Mixing Time Analysis under Gaussian Targets](https://mlsanity.com/paper/2603.18640/) (2026)

## Abstract (arXiv)

Locally adapting parameters within Markov chain Monte Carlo methods while preserving reversibility is notoriously difficult. The success of the No-U-Turn Sampler (NUTS) largely stems from its clever local adaptation of the integration time in Hamiltonian Monte Carlo via a geometric U-turn condition. However, posterior distributions frequently exhibit multi-scale geometries with extreme variations in scale, making it necessary to also adapt the leapfrog integrator's step size locally and dynamically. Despite its practical importance, this problem has remained largely open since the introduction of NUTS by Hoffman and Gelman (2014). To address this issue, we introduce the Within-orbit Adaptive Leapfrog No-U-Turn Sampler (WALNUTS), a generalization of NUTS that adapts the leapfrog step size at fixed intervals of simulated time as the orbit evolves. At each interval, the algorithm selects the largest step size from a dyadic schedule that keeps the energy error below a user-specified threshold. Like NUTS, WALNUTS employs biased progressive state selection to favor states with positions that are further from the initial point along the orbit. Empirical evaluations on multiscale target distributions, including Neal's funnel and the Stock-Watson stochastic volatility time-series model, demonstrate that WALNUTS achieves substantial improvements in sampling efficiency and robustness compared to standard NUTS.