# Interacting Particle Langevin Algorithm for Maximum Marginal Likelihood Estimation arXiv: https://arxiv.org/abs/2303.13429 ยท HTML: https://mlsanity.com/paper/2303.13429/ ## Key facts - Cited by 17 papers in the map (data through 14 Oct 2026). - Builds on 21 of its 84 references that are in the map. - First posted on arXiv 23 Mar 2023. ## Builds on (most cited first, 21 of 21) - [Optimization Methods for Large-Scale Machine Learning](https://mlsanity.com/paper/1606.04838/) (2016) - [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016) - [Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysis](https://mlsanity.com/paper/1702.03849/) (2017) - [Rapid Convergence of the Unadjusted Langevin Algorithm: Isoperimetry Suffices](https://mlsanity.com/paper/1903.08568/) (2019) - [Theoretical guarantees for approximate sampling from smooth and log-concave densities](https://mlsanity.com/paper/1412.7392/) (2014) - [Non-asymptotic convergence analysis for the Unadjusted Langevin Algorithm](https://mlsanity.com/paper/1507.05021/) (2015) - [High-dimensional Bayesian inference via the Unadjusted Langevin Algorithm](https://mlsanity.com/paper/1605.01559/) (2016) - [On the Properties of the Softmax Function with Application in Game Theory and Reinforcement Learning](https://mlsanity.com/paper/1704.00805/) (2017) - [Analysis of Langevin Monte Carlo via convex optimization](https://mlsanity.com/paper/1802.09188/) (2018) - [User-friendly guarantees for the Langevin Monte Carlo with inaccurate gradient](https://mlsanity.com/paper/1710.00095/) (2017) - [Convex Analysis of the Mean Field Langevin Dynamics](https://mlsanity.com/paper/2201.10469/) (2022) - [Mean-Field Langevin Dynamics and Energy Landscape of Neural Networks](https://mlsanity.com/paper/1905.07769/) (2019) - [Nonasymptotic estimates for Stochastic Gradient Langevin Dynamics under local conditions in nonconvex optimization](https://mlsanity.com/paper/1910.02008/) (2019) - [The Tamed Unadjusted Langevin Algorithm](https://mlsanity.com/paper/1710.05559/) (2017) - [Particle algorithms for maximum likelihood training of latent variable models](https://mlsanity.com/paper/2204.12965/) (2022) - [A consensus-based model for global optimization and its mean-field limit](https://mlsanity.com/paper/1604.05648/) (2016) - [Quantitative Propagation of Chaos for SGD in Wide Neural Networks](https://mlsanity.com/paper/2007.06352/) (2020) - [Coin Sampling: Gradient-Based Bayesian Inference without Learning Rates](https://mlsanity.com/paper/2301.11294/) (2023) - [From particle swarm optimization to consensus based optimization: stochastic modeling and mean-field limit](https://mlsanity.com/paper/2012.05613/) (2020) - [Parallel sequential Monte Carlo for stochastic gradient-free nonconvex optimization](https://mlsanity.com/paper/1811.09469/) (2018) - [Consensus-Based Global Optimization with Personal Best](https://mlsanity.com/paper/2005.07084/) (2020) ## Built on by (most cited first, 17 of 17) - [Error bounds for particle gradient descent, and extensions of the log-Sobolev and Talagrand inequalities](https://mlsanity.com/paper/2403.02004/) (2024) - [Tuning-Free Maximum Likelihood Training of Latent Variable Models via Coin Betting](https://mlsanity.com/paper/2305.14916/) (2023) - [Proximal Interacting Particle Langevin Algorithms](https://mlsanity.com/paper/2406.14292/) (2024) - [Kinetic Interacting Particle Langevin Monte Carlo](https://mlsanity.com/paper/2407.05790/) (2024) - [Stochastic Approximation with Biased MCMC for Expectation Maximization](https://mlsanity.com/paper/2402.17870/) (2024) - [Learning Latent Energy-Based Models via Interacting Particle Langevin Dynamics](https://mlsanity.com/paper/2510.12311/) (2025) - [Training Latent Diffusion Models with Interacting Particle Algorithms](https://mlsanity.com/paper/2505.12412/) (2025) - [Uniform-in-time convergence bounds for Persistent Contrastive Divergence algorithms](https://mlsanity.com/paper/2510.01944/) (2025) - [Taming the Interacting Particle Langevin Algorithm: The Superlinear case](https://mlsanity.com/paper/2403.19587/) (2024) - [Learning Latent Variable Models via Jarzynski-adjusted Langevin Algorithm](https://mlsanity.com/paper/2505.18427/) (2025) - [How to beat a Bayesian adversary](https://mlsanity.com/paper/2407.08678/) (2024) - [Fast convergence of the Expectation Maximization algorithm under a logarithmic Sobolev inequality](https://mlsanity.com/paper/2407.17949/) (2024) - [Sparse Techniques for Regression in Deep Gaussian Processes](https://mlsanity.com/paper/2505.11355/) (2025) - [Efficient Stochastic Optimisation via Sequential Monte Carlo](https://mlsanity.com/paper/2601.22003/) (2026) - [Hierarchical Inference and Closure Learning via Adaptive Surrogates for ODEs and PDEs](https://mlsanity.com/paper/2603.03922/) (2026) - [Momentum SVGD-EM for Accelerated Maximum Marginal Likelihood Estimation](https://mlsanity.com/paper/2603.08676/) (2026) - [Particle-based Generalised Stochastic Optimisation](https://mlsanity.com/paper/2608.02844/) (2026) ## Abstract (arXiv) We develop a class of interacting particle systems for implementing a maximum marginal likelihood estimation (MMLE) procedure to estimate the parameters of a latent variable model. We achieve this by formulating a continuous-time interacting particle system which can be seen as a Langevin diffusion over an extended state space of parameters and latent variables. In particular, we prove that the parameter marginal of the stationary measure of this diffusion has the form of a Gibbs measure where number of particles acts as the inverse temperature parameter in classical settings for global optimisation. Using a particular rescaling, we then prove geometric ergodicity of this system and bound the discretisation error in a manner that is uniform in time and does not increase with the number of particles. The discretisation results in an algorithm, termed Interacting Particle Langevin Algorithm (IPLA) which can be used for MMLE. We further prove nonasymptotic bounds for the optimisation error of our estimator in terms of key parameters of the problem, and also extend this result to the case of stochastic gradients covering practical scenarios. We provide numerical experiments to illustrate the empirical behaviour of our algorithm in the context of logistic regression with verifiable assumptions. Our setting provides a straightforward way to implement a diffusion-based optimisation routine compared to more classical approaches such as the Expectation Maximisation (EM) algorithm, and allows for especially explicit nonasymptotic bounds.