mlsanityData to 7 Oct 2026

DG-LMC: A Turn-key and Scalable Synchronous Distributed MCMC Algorithm via Langevin Monte Carlo within Gibbs

arXiv 2106.06300 ↗First posted 11 Jun 2021stat.ME cs.AI cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

Performing reliable Bayesian inference on a big data scale is becoming a keystone in the modern era of machine learning. A workhorse class of methods to achieve this task are Markov chain Monte Carlo (MCMC) algorithms and their design to handle distributed datasets has been the subject of many works. However, existing methods are not completely either reliable or computationally efficient. In this paper, we propose to fill this gap in the case where the dataset is partitioned and stored on computing nodes within a cluster under a master/slaves architecture. We derive a user-friendly centralised distributed MCMC algorithm with provable scaling in high-dimensional settings. We illustrate the relevance of the proposed methodology on both synthetic and real data experiments.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 22 of its 49 references that are in the map. First posted on arXiv 11 Jun 2021.

Builds on

22 of 49 references.

References · 22

  1. 2016Deep Learning with Differential Privacycited by 2,682
  2. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,140
  3. 2015Non-asymptotic convergence analysis for the Unadjusted Langevin Algorithmcited by 195
  4. 2019A Survey on Distributed Machine Learningcited by 187
  5. 2016High-dimensional Bayesian inference via the Unadjusted Langevin Algorithmcited by 144
  6. 2017User-friendly guarantees for the Langevin Monte Carlo with inaccurate gradientcited by 116
  7. 2018The promises and pitfalls of Stochastic Gradient Langevin Dynamicscited by 56
  8. 2015On Markov chain Monte Carlo methods for tall datacited by 50
  9. 2018Split-and-augmented Gibbs sampler - Application to large-scale inference problemscited by 47
  10. 2013Asymptotically Exact, Embarrassingly Parallel MCMCcited by 30
  11. 2018Partitioned Variational Inference: A unified framework encompassing federated and continual learningcited by 29
  12. 2019Efficient MCMC Sampling with Dimension-Free Convergence Rate using ADMM-type Splittingcited by 22
10 more
  1. 2015Distributed Bayesian Learning with Stochastic Natural-gradient Expectation Propagation and the Posterior Servercited by 17
  2. 2014Expectation propagation as a way of life: A framework for Bayesian inference on partitioned datacited by 14
  3. 2016Merging MCMC Subposteriors through Gaussian-Process Approximationscited by 13
  4. 2013Parallelizing MCMC via Weierstrass Samplercited by 11
  5. 2019Embarrassingly parallel MCMC using deep invertible transformationscited by 11
  6. 2015Parallelizing MCMC with Random Partition Treescited by 10
  7. 2016Stochastic Gradient MCMC with Stale Gradientscited by 9
  8. 2019Asymptotically exact data augmentation: models, properties and algorithmscited by 8
  9. 2015Variational consensus Monte Carlocited by 6
  10. 2019Monte Carlo Fusioncited by 3

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 3; 2023: 4; 2024: 2; 2025: 1.

11 papers.

  1. 2022FedPop: A Bayesian Approach for Personalised Federated Learningcited by 35
  2. 2024Regularization by denoising: Bayesian model and Langevin-within-split Gibbs samplingcited by 9
  3. 2023FedHB: Hierarchical Bayesian Federated Learningcited by 8
  4. 2023Adaptive Compression in Federated Learning via Side Informationcited by 6
  5. 2023Federated Variational Inference Methods for Structured Latent Variable Modelscited by 4
  6. 2023Machine Learning and the Future of Bayesian Computationcited by 4
  7. 2021Asynchronous and Distributed Data Augmentation for Massive Data Settingscited by 2
  8. 2022Membership Inference Attacks via Adversarial Examplescited by 2
  9. 2022Federated Averaging Langevin Dynamics: Toward a unified theory and new algorithmscited by 2
  10. 2024Trustworthy Personalized Bayesian Federated Learning via Posterior Fine-Tunecited by 1
  11. 2025Bregman geometry-aware split Gibbs sampling for Bayesian Poisson inverse problemscited by 0