mlsanityData to 7 Oct 2026

Bayesian Federated Model Compression for Communication and Computation Efficiency

arXiv 2404.07532 ↗First posted 11 Apr 2024cs.LG cs.AI cs.DCcited by 1

Not on the map. Why?

Abstract (arXiv)

In this paper, we investigate Bayesian model compression in federated learning (FL) to construct sparse models that can achieve both communication and computation efficiencies. We propose a decentralized Turbo variational Bayesian inference (D-Turbo-VBI) FL framework where we firstly propose a hierarchical sparse prior to promote a clustered sparse structure in the weight matrix. Then, by carefully integrating message passing and VBI with a decentralized turbo framework, we propose the D-Turbo-VBI algorithm which can (i) reduce both upstream and downstream communication overhead during federated training, and (ii) reduce the computational complexity during local inference. Additionally, we establish the convergence property for thr proposed D-Turbo-VBI algorithm. Simulation results show the significant gain of our proposed algorithm over the baselines in reducing communication overhead during federated training and computational complexity of final model.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 19 of its 28 references that are in the map. First posted on arXiv 11 Apr 2024.

Builds on

19 of 28 references.

References · 19

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  4. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
  5. 2020Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Userscited by 336
  6. 2018Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learningcited by 277
  7. 2017Bayesian Compression for Deep Learningcited by 116
  8. 2020Bayesian Bits: Unifying Quantization and Pruningcited by 93
  9. 2016Group Sparse Regularization for Deep Neural Networkscited by 89
  10. 2022Personalized Federated Learning via Variational Bayesian Inferencecited by 68
  11. 2020An Overview of Neural Network Compressioncited by 59
  12. 2020Ternary Compression for Communication-Efficient Federated Learningcited by 57
7 more
  1. 2021A Bayesian Federated Learning Framework with Online Laplace Approximationcited by 41
  2. 2021When to Prune? A Policy towards Early Structural Pruningcited by 38
  3. 2014Block stochastic gradient iteration for convex and nonconvex optimizationcited by 21
  4. 2019Structured Compression by Weight Encryption for Unstructured Pruning and Quantizationcited by 16
  5. 2020Dynamic Sampling and Selective Masking for Communication-Efficient Federated Learningcited by 16
  6. 2023More Communication Does Not Result in Smaller Generalization Error in Federated Learningcited by 10
  7. 2023Structured Bayesian Compression for Deep Neural Networks Based on The Turbo-VBI Approachcited by 2

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Efficient Federated Learning with Encrypted Data Sharing for Data-Heterogeneous Edge Devicescited by 0