mlsanityData to 14 Oct 2026

Subspace Langevin Monte Carlo

arXiv 2412.13928 ↗First posted 18 Dec 2024stat.ML cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Sampling from high-dimensional distributions has wide applications in data science and machine learning but poses significant computational challenges. We introduce Subspace Langevin Monte Carlo (SLMC), a novel and efficient sampling method that generalizes random-coordinate Langevin Monte Carlo and preconditioned Langevin Monte Carlo by projecting the Langevin update onto subsampled eigenblocks of a time-varying preconditioner at each iteration. The advantage of SLMC is its superior adaptability and computational efficiency compared to traditional Langevin Monte Carlo and preconditioned Langevin Monte Carlo. Using coupling arguments, we establish error guarantees for SLMC and demonstrate its practical effectiveness through a few experiments on sampling from ill-conditioned distributions.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 45 of its 76 references that are in the map. First posted on arXiv 18 Dec 2024.

Builds on

45 of 76 references.

References · 45

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  4. 2021Score-based Generative Modeling in Latent Spacecited by 636
  5. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 507
  6. 2018Measuring the Intrinsic Dimension of Objective Landscapescited by 370
  7. 2017Empirical Analysis of the Hessian of Over-Parametrized Neural Networkscited by 290
  8. 2015Coordinate Descent Algorithmscited by 219
  9. 2016Eigenvalues of the Hessian in Deep Learning: Singularity and Beyondcited by 159
  10. 2015A Complete Recipe for Stochastic Gradient MCMCcited by 146
  11. 2016High-dimensional Bayesian inference via the Unadjusted Langevin Algorithmcited by 146
  12. 2018Analysis of Langevin Monte Carlo via convex optimizationcited by 127
33 more
  1. 2018Sampling as optimization in the space of measures: The Langevin dynamics as a composite optimization problemcited by 123
  2. 2017Measuring Sample Quality with Kernelscited by 119
  3. 2017User-friendly guarantees for the Langevin Monte Carlo with inaccurate gradientcited by 116
  4. 2016Relatively-Smooth Convex Optimization by First-Order Methods, and Applicationscited by 108
  5. 2019Theoretical guarantees for sampling and inference in generative models with latent diffusionscited by 97
  6. 2018On the Relation Between the Sharpest Directions of DNN Loss and the SGD Step Lengthcited by 88
  7. 2019Communication-efficient distributed SGD with Sketchingcited by 81
  8. 2021Differential Privacy Dynamics of Langevin Diffusion and Noisy Gradient Descentcited by 73
  9. 2022Improved analysis for a proximal algorithm for samplingcited by 73
  10. 2022Subspace Diffusion Generative Modelscited by 70
  11. 2018Mirrored Langevin Dynamicscited by 61
  12. 2020Wasserstein Control of Mirror Langevin Monte Carlocited by 55
  13. 2020Efficient constrained sampling via the mirror-Langevin algorithmcited by 55
  14. 2020Exponential ergodicity of mirror-Langevin diffusionscited by 37
  15. 2019Scalable Semidefinite Programmingcited by 36
  16. 2024Does SGD really happen in tiny subspaces?cited by 36
  17. 2018Stochastic Quasi-Gradient Methods: Variance Reduction via Jacobian Sketchingcited by 34
  18. 2019RSN: Randomized Subspace Newtoncited by 34
  19. 2018SEGA: Variance Reduction via Gradient Sketchingcited by 29
  20. 2024Sampling in Unit Time with Kernel Fisher-Rao Flowcited by 27
  21. 2020Information Newton's flow: second-order optimization method in probability spacecited by 24
  22. 2020Stochastic Subspace Cubic Newton Methodcited by 21
  23. 2021Efficient Gradient Flows in Sliced-Wasserstein Spacecited by 21
  24. 2020Improving Neural Network Training in Low Dimensional Random Basescited by 19
  25. 2023Sketchy: Memory-efficient Adaptive Regularization with Frequent Directionscited by 17
  26. 2020A stochastic subspace approach to gradient-free optimization in high dimensionscited by 16
  27. 2024Exploring Low-Dimensional Subspaces in Diffusion Models for Controllable Image Editingcited by 15
  28. 2023Optimal Preconditioning and Fisher Adaptive Langevin Samplingcited by 14
  29. 2024Mirror and Preconditioned Gradient Descent in Wasserstein Spacecited by 12
  30. 2015Stochastic Dual Ascent for Solving Linear Systemscited by 8
  31. 2023Scalable Stochastic Gradient Riemannian Langevin Dynamics in Non-Diagonal Metricscited by 8
  32. 2020Langevin Monte Carlo: random coordinate descent and variance reductioncited by 7
  33. 2020Random Coordinate Langevin Monte Carlocited by 7

Built on by

Not cited yet.