# Auxiliary gradient-based sampling algorithms

arXiv: https://arxiv.org/abs/1610.09641 · HTML: https://mlsanity.com/paper/1610.09641/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Oct 2016.

## Built on by (most cited first, 21 of 21)

- [The probability flow ODE is provably fast](https://mlsanity.com/paper/2305.11798/) (2023)
- [Improved analysis for a proximal algorithm for sampling](https://mlsanity.com/paper/2202.06386/) (2022)
- [Faster high-accuracy log-concave sampling via algorithmic warm starts](https://mlsanity.com/paper/2302.10249/) (2023)
- [Bayesian Imaging With Data-Driven Priors Encoded by Neural Networks: Theory, Methods, and Algorithms](https://mlsanity.com/paper/2103.10182/) (2021)
- [Diffusive Gibbs Sampling](https://mlsanity.com/paper/2402.03008/) (2024)
- [Conditional sampling within generative diffusion models](https://mlsanity.com/paper/2409.09650/) (2024)
- [Optimal Preconditioning and Fisher Adaptive Langevin Sampling](https://mlsanity.com/paper/2305.14442/) (2023)
- [Mixing Time of the Proximal Sampler in Relative Fisher Information via Strong Data Processing Inequality](https://mlsanity.com/paper/2502.05623/) (2025)
- [Enhanced gradient-based MCMC in discrete spaces](https://mlsanity.com/paper/2208.00040/) (2022)
- [Mean-Square Analysis of Discretized Ito Diffusions for Heavy-tailed Sampling](https://mlsanity.com/paper/2303.00570/) (2023)
- [Non-Log-Concave and Nonsmooth Sampling via Langevin Monte Carlo Algorithms](https://mlsanity.com/paper/2305.15988/) (2023)
- [Beyond Scores: Proximal Diffusion Models](https://mlsanity.com/paper/2507.08956/) (2025)
- [Operator-Level Quantum Acceleration of Non-Logconcave Sampling](https://mlsanity.com/paper/2505.05301/) (2025)
- [Gaussian Invariant Markov Chain Monte Carlo](https://mlsanity.com/paper/2506.21511/) (2025)
- [Fast-Mixing Markov Chains without Gradients](https://mlsanity.com/paper/2606.27564/) (2026)
- [Auxiliary MCMC and particle Gibbs samplers for parallelisable inference in latent dynamical systems](https://mlsanity.com/paper/2303.00301/) (2023)
- [Markov chain Monte Carlo without evaluating the target: an auxiliary variable approach](https://mlsanity.com/paper/2406.05242/) (2024)
- [Fast Gibbs sampling for the local and global trend Bayesian exponential smoothing model](https://mlsanity.com/paper/2407.00492/) (2024)
- [Preconditioned Discrete-HAMS: A Second-order Irreversible Discrete Sampler](https://mlsanity.com/paper/2507.21982/) (2025)
- [Local Geometric Mixing via Dobrushin Contraction with Applications to Diffusion Path Monte Carlo and the Proximal Sampler](https://mlsanity.com/paper/2609.28338/) (2026)
- [On skew-symmetric distributions and their use in Monte Carlo sampling algorithms: coordinate-free, Gibbs-style and manifold versions of the Barker proposal](https://mlsanity.com/paper/2610.01448/) (2026)

## Abstract (arXiv)

We introduce a new family of MCMC samplers that combine auxiliary variables, Gibbs sampling and Taylor expansions of the target density. Our approach permits the marginalisation over the auxiliary variables yielding marginal samplers, or the augmentation of the auxiliary variables, yielding auxiliary samplers. The well-known Metropolis-adjusted Langevin algorithm (MALA) and preconditioned Crank-Nicolson Langevin (pCNL) algorithm are shown to be special cases. We prove that marginal samplers are superior in terms of asymptotic variance and demonstrate cases where they are slower in computing time compared to auxiliary samplers. In the context of latent Gaussian models we propose new auxiliary and marginal samplers whose implementation requires a single tuning parameter, which can be found automatically during the transient phase. Extensive experimentation shows that the increase in efficiency (measured as effective sample size per unit of computing time) relative to (optimised implementations of) pCNL, elliptical slice sampling and MALA ranges from 10-fold in binary classification problems to 25-fold in log-Gaussian Cox processes to 100-fold in Gaussian process regression, and it is on par with Riemann manifold Hamiltonian Monte Carlo in an example where the latter has the same complexity as the aforementioned algorithms. We explain this remarkable improvement in terms of the way alternative samplers try to approximate the eigenvalues of the target. We introduce a novel MCMC sampling scheme for hyperparameter learning that builds upon the auxiliary samplers. The MATLAB code for reproducing the experiments in the article is publicly available and a Supplement to this article contains additional experiments and implementation details.