mlsanityData to 14 Oct 2026

ELF: Federated Langevin Algorithms with Primal, Dual and Bidirectional Compression

arXiv 2303.04622 ↗First posted 8 Mar 2023stat.ML cs.LG math.OCcited by 4

Not on the map. Why?

Abstract (arXiv)

Federated sampling algorithms have recently gained great popularity in the community of machine learning and statistics. This paper studies variants of such algorithms called Error Feedback Langevin algorithms (ELF). In particular, we analyze the combinations of EF21 and EF21-P with the federated Langevin Monte-Carlo. We propose three algorithms: P-ELF, D-ELF, and B-ELF that use, respectively, primal, dual, and bidirectional compressors. We analyze the proposed methods under Log-Sobolev inequality and provide non-asymptotic convergence guarantees.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 49 of its 82 references that are in the map. First posted on arXiv 8 Mar 2023.

Builds on

49 of 82 references.

References · 49

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  4. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  5. 2016Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Conditioncited by 539
  6. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 387
  7. 2018Sparsified SGD with Memorycited by 376
  8. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 308
  9. 2017Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysiscited by 243
  10. 2018The Convergence of Sparsified Gradient Methodscited by 229
  11. 2019Rapid Convergence of the Unadjusted Langevin Algorithm: Isoperimetry Sufficescited by 227
  12. 2014Theoretical guarantees for approximate sampling from smooth and log-concave densitiescited by 208
37 more
  1. 2015Non-asymptotic convergence analysis for the Unadjusted Langevin Algorithmcited by 197
  2. 2020FedBE: Making Bayesian Model Ensemble Applicable to Federated Learningcited by 176
  3. 2020Better Theory for SGD in the Nonconvex Worldcited by 158
  4. 2016High-dimensional Bayesian inference via the Unadjusted Langevin Algorithmcited by 146
  5. 2017Non-convex Optimization for Machine Learningcited by 133
  6. 2021EF21: A New, Simpler, Theoretically Better, and Practically Faster Error Feedbackcited by 133
  7. 2019Modeling the Dynamics of PDE Systems with Physics-Constrained Deep Auto-Regressive Networkscited by 129
  8. 2018Analysis of Langevin Monte Carlo via convex optimizationcited by 127
  9. 2018Sampling as optimization in the space of measures: The Langevin dynamics as a composite optimization problemcited by 123
  10. 2018Log-concave sampling: Metropolis-Hastings algorithms are fastcited by 118
  11. 2017User-friendly guarantees for the Langevin Monte Carlo with inaccurate gradientcited by 116
  12. 2019Distributed Learning with Compressed Gradient Differencescited by 112
  13. 2019Natural Compression for Distributed Deep Learningcited by 90
  14. 2017Convergence of Langevin MCMC in KL-divergencecited by 88
  15. 2021MARINA: Faster Non-Convex Distributed Learning with Compressioncited by 78
  16. 2018Sharp convergence rates for Langevin dynamics in the nonconvex settingcited by 68
  17. 2019A Systematic Comparison of Bayesian Deep Learning Robustness in Diabetic Retinopathy Taskscited by 62
  18. 2020Bidirectional compression in heterogeneous settings for distributed or federated learning with partial participation: tight convergence guaranteescited by 46
  19. 2020A Better Alternative to Error Feedback for Communication-Efficient Distributed Learningcited by 43
  20. 2022Anticorrelated Noise Injection for Improved Generalizationcited by 42
  21. 2020Federated Generalized Bayesian Learning via Distributed Stein Variational Gradient Descentcited by 35
  22. 2019Proximal Langevin Algorithm: Rapid Convergence Under Isoperimetrycited by 34
  23. 2019A Double Residual Compression Algorithm for Efficient Distributed Learningcited by 31
  24. 2013Langevin diffusions and the Metropolis-adjusted Langevin algorithmcited by 29
  25. 2022EF21-P and Friends: Improved Theoretical Communication Complexity for Distributed Optimization with Bidirectional Compressioncited by 28
  26. 2021QLSD: Quantised Langevin stochastic dynamics for Bayesian federated learningcited by 26
  27. 2018On the Theory of Variance Reduction for Stochastic Gradient Monte Carlocited by 19
  28. 2019Stochastic Proximal Langevin Algorithm: Potential Splitting and Nonasymptotic Ratescited by 17
  29. 2019Bounding the error of discretized Langevin algorithms for non-strongly log-concave targetscited by 16
  30. 2019Nonconvex sampling with the Metropolis-adjusted Langevin algorithmcited by 15
  31. 2021A Bayesian neural network predicts the dissolution of compact planetary systemscited by 15
  32. 2021On Convergence of Federated Averaging Langevin Dynamicscited by 14
  33. 2021Wireless Federated Langevin Monte Carlo: Repurposing Channel Noise for Bayesian Sampling and Privacycited by 10
  34. 2017PBODL : Parallel Bayesian Online Deep Learning for Click-Through Rate Prediction in Tencent Advertising Systemcited by 9
  35. 2022Federated Learning with a Sampling Algorithm under Isoperimetrycited by 4
  36. 2020Penalized Langevin dynamics with vanishing penalty for smooth and log-concave targetscited by 2
  37. 2022Federated Averaging Langevin Dynamics: Toward a unified theory and new algorithmscited by 2

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 3.

4 papers.

  1. 2023Towards a Better Theoretical Understanding of Independent Subnetwork Trainingcited by 6
  2. 2024One-Shot Federated Learning with Bayesian Pseudocoresetscited by 1
  3. 2024Distributed Markov Chain Monte Carlo Sampling based on the Alternating Direction Method of Multiplierscited by 0
  4. 2024Compressed Bayesian Federated Learning for Reliable Passive Radio Sensing in Industrial IoTcited by 0