mlsanityData to 14 Oct 2026

Particle-based Generalised Stochastic Optimisation

arXiv 2608.02844 ↗First posted 3 Aug 2026stat.ML cs.LG stat.COcited by 0

Not on the map. Why?

Abstract (arXiv)

We develop a class of general diffusion-based stochastic particle optimisation methods for loss functions with intractable gradients. Specifically, we consider problems in which the loss gradient is an integral with respect to a parameter-dependent distribution, a structure that includes training generative models, fine-tuning, and learning latent-variable models. We introduce mean-field dynamics and its interacting-particle approximations, which contain several existing algorithms as special cases and provide a route to constructing new methods. Under well-posedness and joint contractivity assumptions, we prove exponential convergence and show that the continuous-time particle system admits a non-asymptotic error bound. We illustrate our framework by developing higher-order Langevin and momentum optimisers and evaluating them on maximum marginal-likelihood estimation and energy-based-model training.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 26 of its 45 references that are in the map. First posted on arXiv 3 Aug 2026.

Builds on

26 of 45 references.

References · 26

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2021How to Train Your Energy-Based Modelscited by 274
  3. 2019Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Modelcited by 148
  4. 2015A Complete Recipe for Stochastic Gradient MCMCcited by 146
  5. 2020Learning Latent Space Energy-Based Prior Modelcited by 104
  6. 2016Efficient Bayesian computation by proximal Markov chain Monte Carlo: when Langevin meets Moreaucited by 84
  7. 2018Convergence and Dynamical Behavior of the ADAM Algorithm for Non-Convex Stochastic Optimizationcited by 49
  8. 2019High-Order Langevin Diffusion Yields an Accelerated MCMC Algorithmcited by 36
  9. 2018A general system of differential equations to model first order adaptive algorithmscited by 28
  10. 2022Particle algorithms for maximum likelihood training of latent variable modelscited by 26
  11. 2024Implicit Diffusion: Efficient Optimization through Stochastic Samplingcited by 24
  12. 2023Interacting Particle Langevin Algorithm for Maximum Marginal Likelihood Estimationcited by 17
14 more
  1. 2024Error bounds for particle gradient descent, and extensions of the log-Sobolev and Talagrand inequalitiescited by 13
  2. 2023Momentum Particle Maximum Likelihoodcited by 12
  3. 2024Proximal Interacting Particle Langevin Algorithmscited by 8
  4. 2024Kinetic Interacting Particle Langevin Monte Carlocited by 7
  5. 2024Modeling AdaGrad, RMSProp, and Adam with Integro-Differential Equationscited by 7
  6. 2021Gradient-Based Markov Chain Monte Carlo for Bayesian Inference With Non-Differentiable Priorscited by 6
  7. 2024Stochastic Approximation with Biased MCMC for Expectation Maximizationcited by 5
  8. 2025Learning Latent Energy-Based Models via Interacting Particle Langevin Dynamicscited by 5
  9. 2021A Unifying and Canonical Description of Measure-Preserving Diffusionscited by 4
  10. 2025Accelerating Langevin Monte Carlo Sampling: A Large Deviations Analysiscited by 4
  11. 2025Training Latent Diffusion Models with Interacting Particle Algorithmscited by 4
  12. 2025Uniform-in-time convergence bounds for Persistent Contrastive Divergence algorithmscited by 4
  13. 2025Learning Latent Variable Models via Jarzynski-adjusted Langevin Algorithmcited by 3
  14. 2026Efficient Stochastic Optimisation via Sequential Monte Carlocited by 1

Built on by

Not cited yet.