# Regime-Switching Langevin Monte Carlo Algorithms

arXiv: https://arxiv.org/abs/2509.00941 · HTML: https://mlsanity.com/paper/2509.00941/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 52 references that are in the map.
- First posted on arXiv 31 Aug 2025.

## Builds on (most cited first, 24 of 24)

- [Cyclical Learning Rates for Training Neural Networks](https://mlsanity.com/paper/1506.01186/) (2015)
- [Snapshot Ensembles: Train 1, get M for free](https://mlsanity.com/paper/1704.00109/) (2017)
- [Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysis](https://mlsanity.com/paper/1702.03849/) (2017)
- [Theoretical guarantees for approximate sampling from smooth and log-concave densities](https://mlsanity.com/paper/1412.7392/) (2014)
- [Non-asymptotic convergence analysis for the Unadjusted Langevin Algorithm](https://mlsanity.com/paper/1507.05021/) (2015)
- [High-dimensional Bayesian inference via the Unadjusted Langevin Algorithm](https://mlsanity.com/paper/1605.01559/) (2016)
- [Underdamped Langevin MCMC: A non-asymptotic analysis](https://mlsanity.com/paper/1707.03663/) (2017)
- [User-friendly guarantees for the Langevin Monte Carlo with inaccurate gradient](https://mlsanity.com/paper/1710.00095/) (2017)
- [Global Convergence of Langevin Dynamics Based Algorithms for Nonconvex Optimization](https://mlsanity.com/paper/1707.06618/) (2017)
- [Convergence of Langevin MCMC in KL-divergence](https://mlsanity.com/paper/1705.09048/) (2017)
- [The Randomized Midpoint Method for Log-Concave Sampling](https://mlsanity.com/paper/1909.05503/) (2019)
- [Sharp convergence rates for Langevin dynamics in the nonconvex setting](https://mlsanity.com/paper/1805.01648/) (2018)
- [Consistency and fluctuations for stochastic gradient Langevin dynamics](https://mlsanity.com/paper/1409.0578/) (2014)
- [On sampling from a log-concave density using kinetic Langevin diffusions](https://mlsanity.com/paper/1807.09382/) (2018)
- [Nonasymptotic estimates for Stochastic Gradient Langevin Dynamics under local conditions in nonconvex optimization](https://mlsanity.com/paper/1910.02008/) (2019)
- [Global Convergence of Stochastic Gradient Hamiltonian Monte Carlo for Non-Convex Stochastic Optimization: Non-Asymptotic Performance Bounds and Momentum-Based Acceleration](https://mlsanity.com/paper/1809.04618/) (2018)
- [On stochastic gradient Langevin dynamics with dependent data streams in the logconcave case](https://mlsanity.com/paper/1812.02709/) (2018)
- [Decentralized Stochastic Gradient Langevin Dynamics and Hamiltonian Monte Carlo](https://mlsanity.com/paper/2007.00590/) (2020)
- [On stochastic gradient Langevin dynamics with dependent data streams: the fully non-convex case](https://mlsanity.com/paper/1905.13142/) (2019)
- [Breaking Reversibility Accelerates Langevin Dynamics for Global Non-Convex Optimization](https://mlsanity.com/paper/1812.07725/) (2018)
- [Exploring loss function topology with cyclical learning rates](https://mlsanity.com/paper/1702.04283/) (2017)
- [Deep Reinforcement Learning using Cyclical Learning Rates](https://mlsanity.com/paper/2008.01171/) (2020)
- [Stochastic gradient descent with random learning rate](https://mlsanity.com/paper/2003.06926/) (2020)
- [Cyclic and Randomized Stepsizes Invoke Heavier Tails in SGD than Constant Stepsize](https://mlsanity.com/paper/2302.05516/) (2023)

## Built on by (most cited first, 3 of 3)

- [Variance Reduction for Stochastic Gradient Generalized Non-reversible Langevin Monte Carlo Algorithms](https://mlsanity.com/paper/2606.28808/) (2026)
- [Reflected Anchored Langevin Algorithms](https://mlsanity.com/paper/2610.09522/) (2026)
- [Penalized Nonreversible Langevin for Constrained Sampling](https://mlsanity.com/paper/2609.25381/) (2026)

## Abstract (arXiv)

Langevin Monte Carlo (LMC) algorithms are popular Markov Chain Monte Carlo (MCMC) methods to sample a target probability distribution, which arises in many applications in machine learning. Inspired by regime-switching stochastic differential equations in the probability literature, we propose and study regime-switching Langevin dynamics (RS-LD) and regime-switching kinetic Langevin dynamics (RS-KLD). Based on their discretizations, we introduce regime-switching Langevin Monte Carlo (RS-LMC) and regime-switching kinetic Langevin Monte Carlo (RS-KLMC) algorithms, which can also be viewed as LMC and KLMC algorithms with random stepsizes. We also propose frictional-regime-switching kinetic Langevin dynamics (FRS-KLD) and its associated algorithm frictional-regime-switching kinetic Langevin Monte Carlo (FRS-KLMC), which can also be viewed as the KLMC algorithm with random frictional coefficients. We provide their 2-Wasserstein non-asymptotic convergence guarantees to the target distribution, and analyze the iteration complexities. Numerical experiments using both synthetic and real data are provided to illustrate the efficiency of our proposed algorithms.