mlsanityData to 7 Oct 2026

FedBIAD: Communication-Efficient and Accuracy-Guaranteed Federated Learning with Bayesian Inference-Based Adaptive Dropout

arXiv 2307.07172 ↗First posted 14 Jul 2023cs.DC cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) emerges as a distributed machine learning paradigm without end-user data transmission, effectively avoiding privacy leakage. Participating devices in FL are usually bandwidth-constrained, and the uplink is much slower than the downlink in wireless networks, which causes a severe uplink communication bottleneck. A prominent direction to alleviate this problem is federated dropout, which drops fractional weights of local models. However, existing federated dropout studies focus on random or ordered dropout and lack theoretical support, resulting in unguaranteed performance. In this paper, we propose Federated learning with Bayesian Inference-based Adaptive Dropout (FedBIAD), which regards weight rows of local models as probability distributions and adaptively drops partial weight rows based on importance indicators correlated with the trend of local training loss. By applying FedBIAD, each client adaptively selects a high-quality dropping pattern with accurate approximations and only transmits parameters of non-dropped weight rows to mitigate uplink costs while improving accuracy. Theoretical analysis demonstrates that the convergence rate of the average generalization error of FedBIAD is minimax optimal up to a squared logarithmic factor. Extensive experiments on image classification and next-word prediction show that compared with status quo approaches, FedBIAD provides 2x uplink reduction with an accuracy increase of up to 2.41% even on non-Independent and Identically Distributed (non-IID) data, which brings up to 72% decrease in training time.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 33 of its 46 references that are in the map. First posted on arXiv 14 Jul 2023.

Builds on

33 of 46 references.

References · 33

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  4. 2016Pointer Sentinel Mixture Modelscited by 2,992
  5. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  6. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  7. 2018LEAF: A Benchmark for Federated Settingscited by 780
  8. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  9. 2018Federated Learning for Mobile Keyboard Predictioncited by 565
  10. 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 561
  11. 2017Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Trainingcited by 471
  12. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
21 more
  1. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
  2. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
  3. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 307
  4. 2017Regularizing and Optimizing LSTM Language Modelscited by 205
  5. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  6. 2018Expanding the Reach of Federated Learning by Reducing Client Resource Requirementscited by 162
  7. 2015A Theoretically Grounded Application of Dropout in Recurrent Neural Networkscited by 161
  8. 2022Personalized Federated Learning via Variational Bayesian Inferencecited by 68
  9. 2017Concentration of tempered posteriors and of their variational approximationscited by 66
  10. 2021Federated Dropout -- A Simple Approach for Enabling Federated Learning on Resource Constrained Devicescited by 56
  11. 2018Posterior Concentration for Sparse Deep Learningcited by 50
  12. 2018Learning Private Neural Language Modeling with Attentive Aggregationcited by 49
  13. 2014Bayesian linear regression with sparse priorscited by 40
  14. 2020Adaptive Federated Dropout: Improving Communication Efficiency and Generalization for Federated Learningcited by 37
  15. 2021Exact Langevin Dynamics with Stochastic Gradientscited by 30
  16. 2019Convergence Rates of Variational Inference in Sparse Deep Learningcited by 27
  17. 2020Pretraining Federated Text Models for Next Word Predictioncited by 22
  18. 2019On the Computation and Communication Complexity of Parallel SGD with Dynamic Batch Sizes for Stochastic Non-Convex Optimizationcited by 14
  19. 2022How to Combine Variational Bayesian Networks in Federated Learningcited by 5
  20. 2013On Fast Dropout and its Applicability to Recurrent Networkscited by 4
  21. 2017Bayesian Sparsification of Recurrent Neural Networkscited by 4

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 2.

4 papers.

  1. 2024Tackling Noisy Clients in Federated Learning with End-to-end Label Correctioncited by 11
  2. 2025FNBench: Benchmarking Robust Federated Learning against Noisy Labelscited by 6
  3. 2024Learnable Sparse Customization in Heterogeneous Edge Computingcited by 2
  4. 2025Traversal Learning: A Lossless And Efficient Distributed Learning Frameworkcited by 0