# Constrained Exploration via Reflected Replica Exchange Stochastic Gradient Langevin Dynamics

arXiv: https://arxiv.org/abs/2405.07839 · HTML: https://mlsanity.com/paper/2405.07839/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 65 references that are in the map.
- First posted on arXiv 13 May 2024.

## Builds on (most cited first, 26 of 26)

- [Stochastic Gradient Descent as Approximate Bayesian Inference](https://mlsanity.com/paper/1704.04289/) (2017)
- [Cyclical Stochastic Gradient MCMC for Bayesian Deep Learning](https://mlsanity.com/paper/1902.03932/) (2019)
- [PySINDy: A comprehensive Python package for robust sparse system identification](https://mlsanity.com/paper/2111.08481/) (2021)
- [Mirrored Langevin Dynamics](https://mlsanity.com/paper/1802.10174/) (2018)
- [Efficient constrained sampling via the mirror-Langevin algorithm](https://mlsanity.com/paper/2010.16212/) (2020)
- [Sampling from a log-concave distribution with Projected Langevin Monte Carlo](https://mlsanity.com/paper/1507.02564/) (2015)
- [Sampling from a log-concave distribution with compact support with proximal Langevin Monte Carlo](https://mlsanity.com/paper/1705.08964/) (2017)
- [On the Convergence of Stochastic Gradient MCMC Algorithms with High-Order Integrators](https://mlsanity.com/paper/1610.06665/) (2016)
- [Non-convex Learning via Replica Exchange Stochastic Gradient MCMC](https://mlsanity.com/paper/2008.05367/) (2020)
- [Beyond Log-concavity: Provable Guarantees for Sampling Multi-modal Distributions using Simulated Tempering Langevin Monte Carlo](https://mlsanity.com/paper/1710.02736/) (2017)
- [Stochastic Runge-Kutta Accelerates Langevin Monte Carlo and Beyond](https://mlsanity.com/paper/1906.07868/) (2019)
- [Projected Stochastic Gradient Langevin Algorithms for Constrained Sampling and Non-Convex Learning](https://mlsanity.com/paper/2012.12137/) (2020)
- [Convergence Rate of Riemannian Hamiltonian Monte Carlo and Faster Polytope Volume Computation](https://mlsanity.com/paper/1710.06261/) (2017)
- [MOTS: Minimax Optimal Thompson Sampling](https://mlsanity.com/paper/2003.01803/) (2020)
- [Sampling with Riemannian Hamiltonian Monte Carlo in a Constrained Space](https://mlsanity.com/paper/2202.01908/) (2022)
- [A Contour Stochastic Gradient Langevin Dynamics Algorithm for Simulations of Multi-modal Distributions](https://mlsanity.com/paper/2010.09800/) (2020)
- [Accelerating Nonconvex Learning via Replica Exchange Langevin Diffusion](https://mlsanity.com/paper/2007.01990/) (2020)
- [Fast Convergence of Langevin Dynamics on Manifold: Geodesics meet Log-Sobolev](https://mlsanity.com/paper/2010.05263/) (2020)
- [Stochastic Quasi-Newton Langevin Monte Carlo](https://mlsanity.com/paper/1602.03442/) (2016)
- [Reflected Schr"odinger Bridge for Constrained Generative Modeling](https://mlsanity.com/paper/2401.03228/) (2024)
- [Sampling in Constrained Domains with Orthogonal-Space Variational Gradient Descent](https://mlsanity.com/paper/2210.06447/) (2022)
- [Accelerating Approximate Thompson Sampling with Underdamped Langevin Monte Carlo](https://mlsanity.com/paper/2401.11665/) (2024)
- [Interacting Contour Stochastic Gradient Langevin Dynamics](https://mlsanity.com/paper/2202.09867/) (2022)
- [Accelerating Hamiltonian Monte Carlo via Chebyshev Integration Time](https://mlsanity.com/paper/2207.02189/) (2022)
- [Non-reversible Parallel Tempering for Deep Posterior Approximation](https://mlsanity.com/paper/2211.10837/) (2022)
- [Unbiased constrained sampling with Self-Concordant Barrier Hamiltonian Monte Carlo](https://mlsanity.com/paper/2210.11925/) (2022)

## Built on by (most cited first, 7 of 7)

- [Non-Reversible Langevin Algorithms for Constrained Sampling](https://mlsanity.com/paper/2501.11743/) (2025)
- [LES-SINDy: Laplace-Enhanced Sparse Identification of Nonlinear Dynamical Systems](https://mlsanity.com/paper/2411.01719/) (2024)
- [Morephy-Net: An Evolutionary Multi-objective Optimization for Replica-Exchange-based Physics-informed Neural Operator Learning Networks](https://mlsanity.com/paper/2509.00663/) (2025)
- [Accelerating Constrained Sampling: A Large Deviations Approach](https://mlsanity.com/paper/2506.07816/) (2025)
- [Decentralized Proximal Stochastic Gradient Langevin Dynamics](https://mlsanity.com/paper/2605.00723/) (2026)
- [BLADE: Bayesian Langevin Active Discovery with Replica Exchange for Identification of Complex Systems](https://mlsanity.com/paper/2503.02983/) (2025)
- [Fast PAC Global Optimization via Restarted Langevin: Exploration, Exploitation, and Degenerate Cooling](https://mlsanity.com/paper/2609.06196/) (2026)

## Abstract (arXiv)

Replica exchange stochastic gradient Langevin dynamics (reSGLD) is an effective sampler for non-convex learning in large-scale datasets. However, the simulation may encounter stagnation issues when the high-temperature chain delves too deeply into the distribution tails. To tackle this issue, we propose reflected reSGLD (r2SGLD): an algorithm tailored for constrained non-convex exploration by utilizing reflection steps within a bounded domain. Theoretically, we observe that reducing the diameter of the domain enhances mixing rates, exhibiting a textitquadratic behavior. Empirically, we test its performance through extensive experiments, including identifying dynamical systems with physical constraints, simulations of constrained multi-modal distributions, and image classification tasks. The theoretical and empirical findings highlight the crucial role of constrained exploration in improving the simulation efficiency.