mlsanityData to 7 Oct 2026

FedBAT: Communication-Efficient Federated Learning via Learnable Binarization

arXiv 2408.03215 ↗First posted 6 Aug 2024cs.LG cs.DCcited by 9

Not on the map. Why?

Abstract (arXiv)

Federated learning is a promising distributed machine learning paradigm that can effectively exploit large-scale data without exposing users' privacy. However, it may incur significant communication overhead, thereby potentially impairing the training efficiency. To address this challenge, numerous studies suggest binarizing the model updates. Nonetheless, traditional methods usually binarize model updates in a post-training manner, resulting in significant approximation errors and consequent degradation in model accuracy. To this end, we propose Federated Binarization-Aware Training (FedBAT), a novel framework that directly learns binary model updates during the local training process, thus inherently reducing the approximation errors. FedBAT incorporates an innovative binarization operator, along with meticulously designed derivatives to facilitate efficient learning. In addition, we establish theoretical guarantees regarding the convergence of FedBAT. Extensive experiments are conducted on four popular datasets. The results show that FedBAT significantly accelerates the convergence and exceeds the accuracy of baselines by up to 9%, even surpassing that of FedAvg in some cases.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 33 of its 40 references that are in the map. First posted on arXiv 6 Aug 2024.

Builds on

33 of 40 references.

References · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  4. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  5. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  6. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
  7. 2019Learned Step Size Quantizationcited by 604
  8. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 571
  9. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
  10. 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 561
  11. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 456
  12. 2020Flower: A Friendly Federated Learning Research Frameworkcited by 449
21 more
  1. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
  2. 2018Sparsified SGD with Memorycited by 373
  3. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
  4. 2019Error Feedback Fixes SignSGD and other Gradient Compression Schemescited by 306
  5. 2018Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learningcited by 277
  6. 2018Expanding the Reach of Federated Learning by Reducing Client Resource Requirementscited by 162
  7. 2021FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learningcited by 140
  8. 2021Adaptive Quantization of Model Updates for Communication-Efficient Federated Learningcited by 60
  9. 2016Loss-aware Binarization of Deep Networkscited by 56
  10. 2019Stochastic Sign Descent Methods: New Algorithms and Better Theorycited by 49
  11. 2020Sparse Weight Activation Trainingcited by 45
  12. 2022ZeroFL: Efficient On-Device Training for Federated Learning with Local Sparsitycited by 45
  13. 2021DAdaQuant: Doubly-adaptive quantization for communication-efficient Federated Learningcited by 40
  14. 2020Adaptive Federated Dropout: Improving Communication Efficiency and Generalization for Federated Learningcited by 37
  15. 2022Sparse Random Networks for Communication-Efficient Federated Learningcited by 32
  16. 2023Bridging Discrete and Backpropagation: Straight-Through and Beyondcited by 31
  17. 2019Distributed Training with Heterogeneous Data: Bridging Median- and Mean-Based Algorithmscited by 29
  18. 2021FedDQ: Communication-Efficient Federated Learning with Descending Quantizationcited by 18
  19. 2023z-SignFedAvg: A Unified Stochastic Sign-based Compression for Federated Learningcited by 17
  20. 2021Communication-Efficient Federated Learning with Binary Neural Networkscited by 13
  21. 2022HideNseek: Federated Lottery Ticket via Server-side Pruning and Sign Supermaskcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 7; 2026: 2.

9 papers.

  1. 2025The Panaceas for Improving Low-Rank Decomposition in Communication-Efficient Federated Learningcited by 5
  2. 2025Beyond Higher Rank: Token-wise Input-Output Projections for Efficient Low-Rank Adaptationcited by 5
  3. 2025Converge Faster, Talk Less: Hessian-Informed Federated Zeroth-Order Optimizationcited by 3
  4. 2025Layer-wise Update Aggregation with Recycling for Communication-Efficient Federated Learningcited by 1
  5. 2025FedBiF: Communication-Efficient Federated Learning via Bits Freezingcited by 1
  6. 2025Personalized Federated Learning with Bidirectional Communication Compression via One-Bit Random Sketchingcited by 1
  7. 2025ParaBlock: Communication-Computation Parallel Block Coordinate Federated Learning for Large Language Modelscited by 0
  8. 2026Federated Learning of Binary Neural Networks: Enabling Low-Cost Inferencecited by 0
  9. 2026FED-FSTQ: Fisher-Guided Token Quantization for Communication-Efficient Federated Fine-Tuning of LLMs on Edge Devicescited by 0