mlsanityData to 7 Oct 2026

FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Data

arXiv 2005.11418 ↗First posted 22 May 2020cs.LG stat.MLcited by 143

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) has become a popular paradigm for learning from distributed data. To effectively utilize data at different devices without moving them to the cloud, algorithms such as the Federated Averaging (FedAvg) have adopted a "computation then aggregation" (CTA) model, in which multiple local updates are performed using local data, before sending the local models to the cloud for aggregation. However, these schemes typically require strong assumptions, such as the local data are identically independent distributed (i.i.d), or the size of the local gradients are bounded. In this paper, we first explicitly characterize the behavior of the FedAvg algorithm, and show that without strong and unrealistic assumptions on the problem structure, the algorithm can behave erratically for non-convex problems (e.g., diverge to infinity). Aiming at designing FL algorithms that are provably fast and require as few assumptions as possible, we propose a new algorithm design strategy from the primal-dual optimization perspective. Our strategy yields a family of algorithms that take the same CTA model as existing algorithms, but they can deal with the non-convex objective, achieve the best possible optimization and communication complexity while being able to deal with both the full batch and mini-batch local computation models. Most importantly, the proposed algorithms are it communication efficient, in the sense that the communication pattern can be adaptive to the level of heterogeneity among the local data. To the best of our knowledge, this is the first algorithmic framework for FL that achieves all the above properties.

Key facts. Cited by 143 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 20; 2022: 30; 2023: 37; 2024: 30; 2025: 18; 2026: 8.

143 papers.

Sort:

By area:

  1. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 565
  2. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  3. 2021A Field Guide to Federated Optimizationcited by 312
  4. 2023Stochastic Controlled Averaging for Federated Learning with Communication Compressioncited by 100
  5. 2021FedCM: Federated Learning with Client-level Momentumcited by 77
  6. 2021On Large-Cohort Training for Federated Learningcited by 73
  7. 2022Where to Begin? On the Impact of Pre-Training and Initialization in Federated Learningcited by 71
  8. 2021Understanding Clipping for Federated Learning: Convergence and Client-Level Differential Privacycited by 66
  9. 2021What Do We Mean by Generalization in Federated Learning?cited by 62
  10. 2022FedVARP: Tackling the Variance Due to Partial Client Participation in Federated Learningcited by 62
  11. 2022Communication-Efficient Stochastic Zeroth-Order Optimization for Federated Learningcited by 59
  12. 2023FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracycited by 49
38 more
  1. 2023Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscapecited by 46
  2. 2022Federated Learning via Inexact ADMMcited by 44
  3. 2021STEM: A Stochastic Two-Sided Momentum Algorithm Achieving Near-Optimal Sample and Communication Complexities for Federated Learningcited by 43
  4. 2021Anarchic Federated Learningcited by 42
  5. 2021Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspectivecited by 38
  6. 2023Momentum Benefits Non-IID Federated Learning Simply and Provablycited by 37
  7. 2022FedADMM: A Federated Primal-Dual Algorithm Allowing Partial Participationcited by 35
  8. 2021Local Adaptivity in Federated Learning: Convergence and Consistencycited by 31
  9. 2022Architecture Agnostic Federated Learning for Neural Networkscited by 30
  10. 2021FedDR -- Randomized Douglas-Rachford Splitting Algorithms for Nonconvex Federated Composite Optimizationcited by 29
  11. 2022Private Non-Convex Federated Learning Without a Trusted Servercited by 26
  12. 2024FeDeRA:Efficient Fine-tuning of Language Models in Federated Learning Leveraging Weight Decompositioncited by 26
  13. 2022FedADMM: A Robust Federated Deep Learning Framework with Adaptivity to System Heterogeneitycited by 25
  14. 2025FedSWA: Improving Generalization in Federated Learning with Highly Heterogeneous Data via Momentum-Based Stochastic Controlled Weight Averagingcited by 25
  15. 2021Personalized Federated Learning for Heterogeneous Clients with Clustered Knowledge Transfercited by 23
  16. 2021The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learningcited by 23
  17. 2023Efficient Federated Learning via Local Adaptive Amended Optimizer with Linear Speedupcited by 23
  18. 2023Bayesian Federated Learning: A Surveycited by 22
  19. 2022AdaBest: Minimizing Client Drift in Federated Learning via Adaptive Bias Estimationcited by 21
  20. 2023Understanding How Consistency Works in Federated Learning via Stage-wise Relaxed Initializationcited by 21
  21. 2022Federated Adversarial Learning: A Framework with Convergence Analysiscited by 17
  22. 2022Federated Self-supervised Learning for Heterogeneous Clientscited by 16
  23. 2023Accelerating Hybrid Federated Learning Convergence under Partial Participationcited by 15
  24. 2023Resilient Constrained Learningcited by 15
  25. 2022FL Games: A Federated Learning Framework for Distribution Shiftscited by 14
  26. 2023Fusion of Global and Local Knowledge for Personalized Federated Learningcited by 14
  27. 2021Efficient Federated Meta-Learning over Multi-Access Wireless Networkscited by 13
  28. 2021Compositional federated learning: Applications in distributionally robust averaging and meta learningcited by 12
  29. 2024Green Federated Learning: A new era of Green Aware AIcited by 12
  30. 2024Non-IID data in Federated Learning: A Survey with Taxonomy, Metrics, Methods, Frameworks and Future Directionscited by 12
  31. 2021Federated Few-Shot Learning with Adversarial Learningcited by 11
  32. 2022Predictive GAN-powered Multi-Objective Optimization for Hybrid Federated Split Learningcited by 11
  33. 2023Over-the-Air Federated Learning and Optimizationcited by 10
  34. 2024Federated Optimization with Doubly Regularized Drift Correctioncited by 10
  35. 2021Efficient and Reliable Overlay Networks for Decentralized Federated Learningcited by 9
  36. 2022Beyond ADMM: A Unified Client-variance-reduced Adaptive Federated Learning Frameworkcited by 9
  37. 2022FedGiA: An Efficient Hybrid Algorithm for Federated Learningcited by 8
  38. 2022Confederated Learning: Federated Learning with Decentralized Edge Serverscited by 7

50 most cited of 143 All 143 (JSON)