mlsanityData to 7 Oct 2026

Learning Rate Free Sampling in Constrained Domains

arXiv 2305.14943 ↗First posted 24 May 2023stat.ML cs.LG stat.MEcited by 3

Not on the map. Why?

Abstract (arXiv)

We introduce a suite of new particle-based algorithms for sampling in constrained domains which are entirely learning rate free. Our approach leverages coin betting ideas from convex optimisation, and the viewpoint of constrained sampling as a mirrored optimisation problem on the space of probability measures. Based on this viewpoint, we also introduce a unifying framework for several existing constrained sampling algorithms, including mirrored Langevin dynamics and mirrored Stein variational gradient descent. We demonstrate the performance of our algorithms on a range of numerical examples, including sampling from targets on the simplex, sampling with fairness constraints, and constrained sampling problems in post-selection inference. Our results indicate that our algorithms achieve competitive performance with existing constrained sampling methods, without the need to tune any hyperparameters.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 48 of its 103 references that are in the map. First posted on arXiv 24 May 2023.

Builds on

48 of 103 references.

References · 48

  1. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 498
  2. 2016A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluationcited by 236
  3. 2019Rapid Convergence of the Unadjusted Langevin Algorithm: Isoperimetry Sufficescited by 224
  4. 2014Theoretical guarantees for approximate sampling from smooth and log-concave densitiescited by 206
  5. 2020Minimax Pareto Fairness: A Multi Objective Perspectivecited by 152
  6. 2016High-dimensional Bayesian inference via the Unadjusted Langevin Algorithmcited by 144
  7. 2016A Kernel Test of Goodness of Fitcited by 143
  8. 2016Coin Betting and Parameter-Free Online Learningcited by 131
  9. 2017Underdamped Langevin MCMC: A non-asymptotic analysiscited by 131
  10. 2018Analysis of Langevin Monte Carlo via convex optimizationcited by 126
  11. 2018Black-Box Reductions for Parameter-free Online Learning in Banach Spacescited by 124
  12. 2018Sampling as optimization in the space of measures: The Langevin dynamics as a composite optimization problemcited by 123
36 more
  1. 2019Maximum Mean Discrepancy Gradient Flowcited by 123
  2. 2017Measuring Sample Quality with Kernelscited by 119
  3. 2017User-friendly guarantees for the Langevin Monte Carlo with inaccurate gradientcited by 116
  4. 2013Exact post-selection inference, with application to the lassocited by 90
  5. 2019On the geometry of Stein variational gradient descentcited by 82
  6. 2020SVGD as a kernelized Wasserstein gradient flow of the chi-squared divergencecited by 64
  7. 2022Towards a Theory of Non-Log-Concave Sampling: First-Order Stationarity Guarantees for Langevin Monte Carlocited by 63
  8. 2018Mirrored Langevin Dynamicscited by 61
  9. 2018On sampling from a log-concave density using kinetic Langevin diffusionscited by 61
  10. 2020A Non-Asymptotic Analysis for Stein Variational Gradient Descentcited by 61
  11. 2021Kernel Stein Discrepancy Descentcited by 57
  12. 2020Wasserstein Control of Mirror Langevin Monte Carlocited by 55
  13. 2020On the Convergence of Langevin Monte Carlo: The Interplay between Tail Growth and Smoothnesscited by 55
  14. 2020Efficient constrained sampling via the mirror-Langevin algorithmcited by 55
  15. 2018A Stein variational Newton methodcited by 49
  16. 2017Message Passing Stein Variational Gradient Descentcited by 47
  17. 2017Stein Variational Policy Gradientcited by 42
  18. 2020Accuracy and Fairness Trade-offs in Machine Learning: A Stochastic Multi-Objective Approachcited by 39
  19. 2020Exponential ergodicity of mirror-Langevin diffusionscited by 37
  20. 2021The Mirror Langevin Algorithm Converges with Vanishing Biascited by 37
  21. 2020Projected Stein Variational Gradient Descentcited by 33
  22. 2017Convergence Rate of Riemannian Hamiltonian Monte Carlo and Faster Polytope Volume Computationcited by 32
  23. 2019Proximal Langevin Algorithm: Rapid Convergence Under Isoperimetrycited by 31
  24. 2022A Finite-Particle Convergence Rate for Stein Variational Gradient Descentcited by 22
  25. 2021A Convergence Theory for SVGD in the Population Limit under Talagrand's Inequality T1cited by 21
  26. 2021Sampling with Mirrored Stein Operatorscited by 17
  27. 2022Sampling with Mollified Interaction Energy Descentcited by 17
  28. 2020Fast Convergence of Langevin Dynamics on Manifold: Geodesics meet Log-Sobolevcited by 16
  29. 2022Convergence of Stein Variational Gradient Descent under a Weaker Smoothness Conditioncited by 16
  30. 2016Stochastic Quasi-Newton Langevin Monte Carlocited by 14
  31. 2022Sampling in Constrained Domains with Orthogonal-Space Variational Gradient Descentcited by 11
  32. 2022Deterministic Langevin Monte Carlo with Normalizing Flows for Bayesian Inferencecited by 10
  33. 2023Coin Sampling: Gradient-Based Bayesian Inference without Learning Ratescited by 8
  34. 2020Random Coordinate Langevin Monte Carlocited by 7
  35. 2020Random Coordinate Underdamped Langevin Monte Carlocited by 6
  36. 2022A Note on the Convergence of Mirrored Stein Variational Gradient Descent under (L0,L1)-Smoothness Conditioncited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 0; 2026: 2.

3 papers.

  1. 2024Learning-Rate-Free Stochastic Optimization over Riemannian Manifoldscited by 1
  2. 2026Analyzing and Improving Diffusion Models for Time-Series Data Imputation: A Proximal Recursion Perspectivecited by 0
  3. 2026Difference of Convex Programming in the Wasserstein Space with Applications to MMD Optimizationcited by 0