mlsanityData to 7 Oct 2026

FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization

arXiv 1909.13014 ↗First posted 28 Sep 2019cs.LG cs.DC math.OCcited by 349

Not on the map. Why?

Abstract (arXiv)

Federated learning is a distributed framework according to which a model is trained over a set of devices, while keeping data localized. This framework faces several systems-oriented challenges which include (i) communication bottleneck since a large number of devices upload their local updates to a parameter server, and (ii) scalability as the federated network consists of millions of devices. Due to these systems challenges as well as issues related to statistical heterogeneity of data and privacy concerns, designing a provably efficient federated learning method is of significant importance yet it remains challenging. In this paper, we present FedPAQ, a communication-efficient Federated Learning method with Periodic Averaging and Quantization. FedPAQ relies on three key features: (1) periodic averaging where models are updated locally at devices and only periodically averaged at the server; (2) partial device participation where only a fraction of devices participate in each round of the training; and (3) quantized message-passing where the edge nodes quantize their updates before uploading to the parameter server. These features address the communications and scalability challenges in federated learning. We also show that FedPAQ achieves near-optimal theoretical guarantees for strongly convex and non-convex loss functions and empirically demonstrate the communication-computation tradeoff provided by our method.

Key facts. Cited by 349 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 61; 2022: 79; 2023: 63; 2024: 68; 2025: 48; 2026: 30.

349 papers.

Sort:

By area:

  1. 2021Personalized Federated Learning using Hypernetworkscited by 210
  2. 2021FedKD: Communication Efficient Federated Learning via Knowledge Distillationcited by 185
  3. 2021FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learningcited by 140
  4. 2021Distributed Learning in Wireless Networks: Recent Progress and Future Challengescited by 125
  5. 2021Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradientscited by 122
  6. 2022FedALA: Adaptive Local Aggregation for Personalized Federated Learningcited by 121
  7. 2023Stochastic Controlled Averaging for Federated Learning with Communication Compressioncited by 100
  8. 2022FedPrompt: Communication-Efficient and Privacy Preserving Prompt Tuning in Federated Learningcited by 86
  9. 2021Personalized Federated Learning with Gaussian Processescited by 76
  10. 2022Personalized Federated Learning via Variational Bayesian Inferencecited by 68
  11. 2022FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learningcited by 65
  12. 2021LightSecAgg: a Lightweight and Versatile Design for Secure Aggregation in Federated Learningcited by 63
38 more
  1. 2021Adaptive Quantization of Model Updates for Communication-Efficient Federated Learningcited by 60
  2. 2021Federated Multi-Armed Banditscited by 59
  3. 2022Communication-Efficient Adaptive Federated Learningcited by 55
  4. 2021Adapt to Adaptation: Learning Personalization for Cross-Silo Federated Learningcited by 53
  5. 2021Hierarchical Federated Learning with Quantization: Convergence Analysis and System Designcited by 47
  6. 2021Cluster-driven Graph Federated Learning over Multiple Domainscited by 43
  7. 2022A Survey on Heterogeneous Federated Learningcited by 42
  8. 2021Device Sampling for Heterogeneous Federated Learning: Theory, Algorithms, and Implementationcited by 40
  9. 2021Communication Efficiency in Federated Learning: Achievements and Challengescited by 40
  10. 2021DAdaQuant: Doubly-adaptive quantization for communication-efficient Federated Learningcited by 40
  11. 2023Federated Learning for Computationally-Constrained Heterogeneous Devices: A Surveycited by 40
  12. 2021Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspectivecited by 38
  13. 2023Efficient Personalized Federated Learning via Sparse Model-Adaptationcited by 38
  14. 2021Quantized Federated Learning under Transmission Delay and Outage Constraintscited by 37
  15. 2022Gradient Obfuscation Gives a False Sense of Security in Federated Learningcited by 37
  16. 2023Momentum Benefits Non-IID Federated Learning Simply and Provablycited by 37
  17. 2021Layer-wise Adaptive Model Aggregation for Scalable Federated Learningcited by 35
  18. 2022FedADMM: A Federated Primal-Dual Algorithm Allowing Partial Participationcited by 35
  19. 2021Dynamic Scheduling for Over-the-Air Federated Edge Learning with Energy Constraintscited by 34
  20. 2022SoteriaFL: A Unified Framework for Private Federated Learning with Communication Compressioncited by 34
  21. 2022Joint Privacy Enhancement and Quantization in Federated Learningcited by 34
  22. 2024Advances and Open Challenges in Federated Foundation Modelscited by 33
  23. 2022Sparse Random Networks for Communication-Efficient Federated Learningcited by 32
  24. 2024Relaxed Contrastive Learning for Federated Learningcited by 32
  25. 2022Knowledge Distillation for Federated Learning: a Practical Guidecited by 31
  26. 2022FedSynth: Gradient Compression via Synthetic Data in Federated Learningcited by 30
  27. 2023Why Batch Normalization Damage Federated Learning on Non-IID Data?cited by 29
  28. 2024A Survey on Efficient Federated Learning Methods for Foundation Model Trainingcited by 29
  29. 2022A Multi-agent Reinforcement Learning Approach for Efficient Client Selection in Federated Learningcited by 28
  30. 2022Personalizing Federated Medical Image Segmentation via Local Calibrationcited by 28
  31. 2021Resource-Efficient Federated Learningcited by 26
  32. 2024Federated LoRA with Sparse Communicationcited by 26
  33. 2021Toward Communication Efficient Adaptive Gradient Methodcited by 25
  34. 2022Scaling Language Model Size in Cross-Device Federated Learningcited by 25
  35. 2023Federated Temporal Difference Learning with Linear Function Approximation under Environmental Heterogeneitycited by 25
  36. 2023Federated Domain Generalization: A Surveycited by 25
  37. 2022Communication-Efficient Federated Learning with Accelerated Client Gradientcited by 24
  38. 2023Green Edge AI: A Contemporary Surveycited by 24

50 most cited of 349 All 349 (JSON)