mlsanityData to 7 Oct 2026

FedSplit: An algorithmic framework for fast federated optimization

arXiv 2005.05238 ↗First posted 11 May 2020cs.LG math.OC stat.MLcited by 108

Not on the map. Why?

Abstract (arXiv)

Motivated by federated learning, we consider the hub-and-spoke model of distributed optimization in which a central authority coordinates the computation of a solution among many agents while limiting communication. We first study some past procedures for federated optimization, and show that their fixed points need not correspond to stationary points of the original optimization problem, even in simple convex settings with deterministic updates. In order to remedy these issues, we introduce FedSplit, a class of algorithms based on operator splitting procedures for solving distributed convex minimization with additive structure. We prove that these procedures have the correct fixed points, corresponding to optima of the original optimization problem, and we characterize their convergence rates under different settings. Our theory shows that these methods are provably robust to inexact computation of intermediate local quantities. We complement our theory with some simple experiments that demonstrate the benefits of our methods in practice.

Key facts. Cited by 108 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 27; 2022: 26; 2023: 24; 2024: 20; 2025: 7; 2026: 4.

108 papers.

Sort:

By area:

  1. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  2. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  3. 2021Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradientscited by 122
  4. 2021Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applicationscited by 96
  5. 2021FedCM: Federated Learning with Client-level Momentumcited by 77
  6. 2021On Large-Cohort Training for Federated Learningcited by 73
  7. 2021What Do We Mean by Generalization in Federated Learning?cited by 62
  8. 2022FedVARP: Tackling the Variance Due to Partial Client Participation in Federated Learningcited by 62
  9. 2022FedNest: Federated Bilevel, Minimax, and Compositional Optimizationcited by 60
  10. 2022Communication-Efficient Stochastic Zeroth-Order Optimization for Federated Learningcited by 59
  11. 2022FedAvg with Fine Tuning: Local Updates Lead to Representation Learningcited by 55
  12. 2021Convergence and Accuracy Trade-Offs in Federated Learning and Meta-Learningcited by 52
38 more
  1. 2022Federated Learning via Decentralized Dataset Distillation in Resource-Constrained Edge Environmentscited by 49
  2. 2023Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscapecited by 46
  3. 2021Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspectivecited by 38
  4. 2022On Convergence of FedProx: Local Dissimilarity Invariant Bounds, Non-smoothness and Beyondcited by 37
  5. 2022FedADMM: A Federated Primal-Dual Algorithm Allowing Partial Participationcited by 35
  6. 2022SoteriaFL: A Unified Framework for Private Federated Learning with Communication Compressioncited by 34
  7. 2021Local Adaptivity in Federated Learning: Convergence and Consistencycited by 31
  8. 2022Architecture Agnostic Federated Learning for Neural Networkscited by 30
  9. 2021FedDR -- Randomized Douglas-Rachford Splitting Algorithms for Nonconvex Federated Composite Optimizationcited by 29
  10. 2024Finite-Time Analysis of On-Policy Heterogeneous Federated Reinforcement Learningcited by 29
  11. 2023Understanding Generalization of Federated Learning via Stability: Heterogeneity Matterscited by 27
  12. 2023Federated Temporal Difference Learning with Linear Function Approximation under Environmental Heterogeneitycited by 25
  13. 2021Fed-ensemble: Improving Generalization through Model Ensembling in Federated Learningcited by 23
  14. 2021Personalized Federated Learning for Heterogeneous Clients with Clustered Knowledge Transfercited by 23
  15. 2021The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learningcited by 23
  16. 2022Accelerated Federated Learning with Decoupled Adaptive Optimizationcited by 23
  17. 2023Efficient Federated Learning via Local Adaptive Amended Optimizer with Linear Speedupcited by 23
  18. 2022AdaBest: Minimizing Client Drift in Federated Learning via Adaptive Bias Estimationcited by 21
  19. 2022Mitigating Data Heterogeneity in Federated Learning with Data Augmentationcited by 21
  20. 2021Exploiting Heterogeneity in Robust Federated Best-Arm Identificationcited by 20
  21. 2022Proportional Fairness in Federated Learningcited by 20
  22. 2023Privacy and Efficiency of Communications in Federated Split Learningcited by 20
  23. 2022Communication Acceleration of Local Gradient Methods via an Accelerated Primal-Dual Algorithm with Inexact Proxcited by 19
  24. 2022Federated Self-supervised Learning for Heterogeneous Clientscited by 16
  25. 2021On Bridging Generic and Personalized Federated Learning for Image Classificationcited by 15
  26. 2022FedSysID: A Federated Approach to Sample-Efficient System Identificationcited by 15
  27. 2021On Convergence of Federated Averaging Langevin Dynamicscited by 14
  28. 2022Anchor Sampling for Federated Learning with Partial Client Participationcited by 14
  29. 2023SimFBO: Towards Simple, Flexible and Communication-efficient Federated Bilevel Learningcited by 14
  30. 2021A Non-parametric View of FedAvg and FedProx: Beyond Stationary Pointscited by 13
  31. 2022Communication Efficient Federated Learning for Generalized Linear Banditscited by 13
  32. 2022Federated Learning on Riemannian Manifoldscited by 13
  33. 2023Communication-Efficient Federated Hypergradient Computation via Aggregated Iterative Differentiationcited by 13
  34. 2021Edge-Native Intelligence for 6G Communications Driven by Federated Learning: A Survey of Trends and Challengescited by 12
  35. 2021FLIX: A Simple and Communication-Efficient Alternative to Local Methods in Federated Learningcited by 12
  36. 2024The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communicationcited by 12
  37. 2022Over-the-Air Federated Learning via Second-Order Optimizationcited by 11
  38. 2023Over-the-Air Federated Learning and Optimizationcited by 10

50 most cited of 108 All 108 (JSON)