mlsanityData to 7 Oct 2026

FedBCD:Communication-Efficient Accelerated Block Coordinate Gradient Descent for Federated Learning

arXiv 2603.05116 ↗First posted 5 Mar 2026cs.LG cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

Although Federated Learning has been widely studied in recent years, there are still high overhead expenses in each communication round for large-scale models such as Vision Transformer. To lower the communication complexity, we propose a novel Federated Block Coordinate Gradient Descent (FedBCGD) method for communication efficiency. The proposed method splits model parameters into several blocks, including a shared block and enables uploading a specific parameter block by each client, which can significantly reduce communication overhead. Moreover, we also develop an accelerated FedBCGD algorithm (called FedBCGD+) with client drift control and stochastic variance reduction. To the best of our knowledge, this paper is the first work on parameter block communication for training large-scale deep models. We also provide the convergence analysis for the proposed algorithms. Our theoretical results show that the communication complexities of our algorithms are a factor 1/N lower than those of existing methods, where N is the number of parameter blocks, and they enjoy much faster convergence than their counterparts. Empirical results indicate the superiority of the proposed algorithms compared to state-of-the-art algorithms. The code is available at https://github.com/junkangLiu0/FedBCGD.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 31 of its 50 references that are in the map. First posted on arXiv 5 Mar 2026.

Builds on

31 of 50 references.

References · 31

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  6. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  7. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
  8. 2020Adaptive Federated Optimizationcited by 1,036
  9. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  10. 2020The Future of Digital Health with Federated Learningcited by 625
  11. 2014Convex Optimization: Algorithms and Complexitycited by 569
  12. 2016Federated Optimization: Distributed Machine Learning for On-Device Intelligencecited by 558
19 more
  1. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
  2. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
  3. 2019Momentum-Based Variance Reduction in Non-Convex SGDcited by 302
  4. 2017Sparse Communication for Distributed Gradient Descentcited by 243
  5. 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
  6. 2015Coordinate Descent Algorithmscited by 218
  7. 2022FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correctioncited by 181
  8. 2020Mime: Mimicking Centralized Stochastic Algorithms in Federated Learningcited by 152
  9. 2021Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradientscited by 122
  10. 2020Local SGD: Unified Theory and New Efficient Methodscited by 88
  11. 2020Differentially Private Secure Multi-Party Computation for Federated Learning in Financial Applicationscited by 58
  12. 2021STEM: A Stochastic Two-Sided Momentum Algorithm Achieving Near-Optimal Sample and Communication Complexities for Federated Learningcited by 43
  13. 2018Global Convergence of Block Coordinate Descent in Deep Learningcited by 38
  14. 2021Bias-Variance Reduced Local SGD for Less Heterogeneous Federated Learningcited by 30
  15. 2023Why (and When) does Local SGD Generalize Better than SGD?cited by 26
  16. 2013Distributed Coordinate Descent Method for Learning with Big Datacited by 23
  17. 2025FedAdamW: A Communication-Efficient Optimizer with Convergence and Generalization Guarantees for Federated Large Modelscited by 15
  18. 2026FedNSAM:Consistency of Local and Global Flatness for Federated Learningcited by 7
  19. 2014A distributed block coordinate descent method for training l1 regularized linear classifierscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026Deconstructing the Failure of Ideal Noise Correction: A Three-Pillar Diagnosiscited by 1
  2. 2026SeFoRA: Sketch-Aggregated Federated Low-Rank Adaptation with Heterogeneous Client Rankscited by 1
  3. 2026PDD: Manifold-Prior Diverse Distillation for Medical Anomaly Detectioncited by 0
  4. 2026From Selection to Scheduling: Federated Geometry-Aware Correction Makes Exemplar Replay Work Better under Continual Dynamic Heterogeneitycited by 0
  5. 2026CTP-FL: Common-Trajectory Gradient Prediction for Federated Learningcited by 0
  6. 2026FedMIX-P: Mixing Local and Global Preconditioners for Federated Vision and Language Model Trainingcited by 0
  7. 2026FedLore: Communication and Memory Efficient Federated Learning via Shared Gradient Low-Rank Projectioncited by 0