# Breaking Reversibility Accelerates Langevin Dynamics for Global Non-Convex Optimization

arXiv: https://arxiv.org/abs/1812.07725 · HTML: https://mlsanity.com/paper/1812.07725/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Dec 2018.

## Built on by (most cited first, 12 of 12)

- [Asymmetric Heavy Tails and Implicit Bias in Gaussian Noise Injections](https://mlsanity.com/paper/2102.07006/) (2021)
- [Kinetic Langevin MCMC Sampling Without Gradient Lipschitz Continuity -- the Strongly Convex Case](https://mlsanity.com/paper/2301.08039/) (2023)
- [Non-Reversible Langevin Algorithms for Constrained Sampling](https://mlsanity.com/paper/2501.11743/) (2025)
- [Neural Sampling in Hierarchical Exponential-family Energy-based Models](https://mlsanity.com/paper/2310.08431/) (2023)
- [Accelerating Langevin Monte Carlo Sampling: A Large Deviations Analysis](https://mlsanity.com/paper/2503.19066/) (2025)
- [Revisiting the Effects of Stochasticity for Hamiltonian Samplers](https://mlsanity.com/paper/2106.16200/) (2021)
- [Regime-Switching Langevin Monte Carlo Algorithms](https://mlsanity.com/paper/2509.00941/) (2025)
- [Accelerating Constrained Sampling: A Large Deviations Approach](https://mlsanity.com/paper/2506.07816/) (2025)
- [Decentralized Proximal Stochastic Gradient Langevin Dynamics](https://mlsanity.com/paper/2605.00723/) (2026)
- [Variance Reduction for Stochastic Gradient Generalized Non-reversible Langevin Monte Carlo Algorithms](https://mlsanity.com/paper/2606.28808/) (2026)
- [Contractive kinetic Langevin samplers beyond global Lipschitz continuity](https://mlsanity.com/paper/2509.12031/) (2025)
- [Improved Analysis for Hessian-free High-resolution Monte Carlo Sampling](https://mlsanity.com/paper/2608.25052/) (2026)

## Abstract (arXiv)

Langevin dynamics (LD) has been proven to be a powerful technique for optimizing a non-convex objective as an efficient algorithm to find local minima while eventually visiting a global minimum on longer time-scales. LD is based on the first-order Langevin diffusion which is reversible in time. We study two variants that are based on non-reversible Langevin diffusions: the underdamped Langevin dynamics (ULD) and the Langevin dynamics with a non-symmetric drift (NLD). Adopting the techniques of Tzen, Liang and Raginsky (2018) for LD to non-reversible diffusions, we show that for a given local minimum that is within an arbitrary distance from the initialization, with high probability, either the ULD trajectory ends up somewhere outside a small neighborhood of this local minimum within a recurrence time which depends on the smallest eigenvalue of the Hessian at the local minimum or they enter this neighborhood by the recurrence time and stay there for a potentially exponentially long escape time. The ULD algorithms improve upon the recurrence time obtained for LD in Tzen, Liang and Raginsky (2018) with respect to the dependency on the smallest eigenvalue of the Hessian at the local minimum. Similar result and improvement are obtained for the NLD algorithm. We also show that non-reversible variants can exit the basin of attraction of a local minimum faster in discrete time when the objective has two local minima separated by a saddle point and quantify the amount of improvement. Our analysis suggests that non-reversible Langevin algorithms are more efficient to locate a local minimum as well as exploring the state space. Our analysis is based on the quadratic approximation of the objective around a local minimum. As a by-product of our analysis, we obtain optimal mixing rates for quadratic objectives in the 2-Wasserstein distance for two non-reversible Langevin algorithms we consider.