mlsanityData to 7 Oct 2026

QLSD: Quantised Langevin stochastic dynamics for Bayesian federated learning

arXiv 2106.00797 ↗First posted 1 Jun 2021cs.LG cs.AI stat.COcited by 26

Not on the map. Why?

Abstract (arXiv)

The objective of Federated Learning (FL) is to perform statistical inference for data which are decentralised and stored locally on networked clients. FL raises many constraints which include privacy and data ownership, communication overhead, statistical heterogeneity, and partial client participation. In this paper, we address these problems in the framework of the Bayesian paradigm. To this end, we propose a novel federated Markov Chain Monte Carlo algorithm, referred to as Quantised Langevin Stochastic Dynamics which may be seen as an extension to the FL setting of Stochastic Gradient Langevin Dynamics, which handles the communication bottleneck using gradient compression. To improve performance, we then introduce variance reduction techniques, which lead to two improved versions coined textttQLSDstar and textttQLSD++. We give both non-asymptotic and asymptotic convergence guarantees for the proposed algorithms. We illustrate their performances using various Bayesian Federated Learning benchmarks.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 39 of its 79 references that are in the map. First posted on arXiv 1 Jun 2021.

Builds on

39 of 79 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2017On Calibration of Modern Neural Networkscited by 4,860
  4. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  5. 2019Advances and Open Problems in Federated Learningcited by 3,021
  6. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  7. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  8. 2018LEAF: A Benchmark for Federated Settingscited by 780
  9. 2017Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Trainingcited by 471
  10. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
  11. 2018Sparsified SGD with Memorycited by 373
  12. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 307
27 more
  1. 2017Sparse Communication for Distributed Gradient Descentcited by 243
  2. 2014Theoretical guarantees for approximate sampling from smooth and log-concave densitiescited by 206
  3. 2020FedBE: Making Bayesian Model Ensemble Applicable to Federated Learningcited by 176
  4. 2020Federated Learning with Compression: Unified Analysis and Sharp Guaranteescited by 159
  5. 2016QSGD: Communication-Efficient SGD via Gradient Quantization and Encodingcited by 147
  6. 2016High-dimensional Bayesian inference via the Unadjusted Langevin Algorithmcited by 144
  7. 2020Uncertainty Quantification and Deep Ensemblescited by 138
  8. 2018Analysis of Langevin Monte Carlo via convex optimizationcited by 126
  9. 2017User-friendly guarantees for the Langevin Monte Carlo with inaccurate gradientcited by 116
  10. 2019Subspace Inference for Bayesian Deep Learningcited by 94
  11. 2017Bridging the Gap between Constant Step Size Stochastic Gradient Descent and Markov Chainscited by 84
  12. 2019Variational Federated Multi-Task Learningcited by 69
  13. 2019Stochastic Distributed Learning with Gradient Quantization and Variance Reductioncited by 64
  14. 2018The promises and pitfalls of Stochastic Gradient Langevin Dynamicscited by 56
  15. 2020Bidirectional compression in heterogeneous settings for distributed or federated learning with partial participation: tight convergence guaranteescited by 45
  16. 2020Federated Generalized Bayesian Learning via Distributed Stein Variational Gradient Descentcited by 34
  17. 2013Asymptotically Exact, Embarrassingly Parallel MCMCcited by 30
  18. 2018Partitioned Variational Inference: A unified framework encompassing federated and continual learningcited by 29
  19. 2017Control Variates for Stochastic Gradient MCMCcited by 25
  20. 2019Efficient MCMC Sampling with Dimension-Free Convergence Rate using ADMM-type Splittingcited by 22
  21. 2015Distributed Bayesian Learning with Stochastic Natural-gradient Expectation Propagation and the Posterior Servercited by 17
  22. 2020Encoding the latent posterior of Bayesian Neural Networks for uncertainty quantificationcited by 17
  23. 2020Shuffled Model of Federated Learning: Privacy, Communication and Accuracy Trade-offscited by 15
  24. 2016Merging MCMC Subposteriors through Gaussian-Process Approximationscited by 13
  25. 2013Parallelizing MCMC via Weierstrass Samplercited by 11
  26. 2015Parallelizing MCMC with Random Partition Treescited by 10
  27. 2021Channel-Driven Monte Carlo Sampling for Bayesian Distributed Learning in Wireless Data Centerscited by 3

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 7; 2023: 7; 2024: 10.

26 papers.

Sort:

By area:

  1. 2022FedPop: A Bayesian Approach for Personalised Federated Learningcited by 35
  2. 2023Bayesian Federated Learning: A Surveycited by 22
  3. 2021Forget-SVGD: Particle-Based Bayesian Federated Unlearningcited by 14
  4. 2021On Convergence of Federated Averaging Langevin Dynamicscited by 14
  5. 2022Parallel MCMC Without Embarrassing Failurescited by 11
  6. 2023Compression with Exact Error Distribution for Federated Learningcited by 9
  7. 2023Scalable Stochastic Gradient Riemannian Langevin Dynamics in Non-Diagonal Metricscited by 8
  8. 2023Adaptive Compression in Federated Learning via Side Informationcited by 6
  9. 2022Federated Bayesian Neural Regression: A Scalable Global Federated Gaussian Processcited by 5
  10. 2022Federated Learning with a Sampling Algorithm under Isoperimetrycited by 4
  11. 2023Federated Variational Inference Methods for Structured Latent Variable Modelscited by 4
  12. 2023ELF: Federated Langevin Algorithms with Primal, Dual and Bidirectional Compressioncited by 4
14 more
  1. 2024FedSI: Federated Subnetwork Inference for Efficient Uncertainty Quantificationcited by 4
  2. 2024Bayesian Federated Learning with Hamiltonian Monte Carlo: Algorithm and Theorycited by 4
  3. 2024Embarrassingly Parallel GFlowNetscited by 3
  4. 2022Membership Inference Attacks via Adversarial Examplescited by 2
  5. 2022Federated Averaging Langevin Dynamics: Toward a unified theory and new algorithmscited by 2
  6. 2023Bayesian Personalized Federated Learning with Shared and Personalized Uncertainty Representationscited by 2
  7. 2022Content Popularity Prediction Based on Quantized Federated Bayesian Learning in Fog Radio Access Networkscited by 1
  8. 2024Trustworthy Personalized Bayesian Federated Learning via Posterior Fine-Tunecited by 1
  9. 2024One-Shot Federated Learning with Bayesian Pseudocoresetscited by 1
  10. 2024Distribution-Aware Mean Estimation under User-level Local Differential Privacycited by 1
  11. 2024Compressed Bayesian Federated Learning for Reliable Passive Radio Sensing in Industrial IoTcited by 0
  12. 2024FedLog: Personalized Federated Classification with Less Communication and More Flexibilitycited by 0
  13. 2024Personalizing Low-Rank Bayesian Neural Networks Via Federated Learningcited by 0
  14. 2024TRAIL: Trust-Aware Client Scheduling for Semi-Decentralized Federated Learningcited by 0