An Operator Splitting View of Federated Learning
Not on the map. Why?
Abstract (arXiv)
Over the past few years, the federated learning (textttFL) community has witnessed a proliferation of new textttFL algorithms. However, our understating of the theory of textttFL is still fragmented, and a thorough, formal comparison of these algorithms remains elusive. Motivated by this gap, we show that many of the existing textttFL algorithms can be understood from an operator splitting point of view. This unification allows us to compare different algorithms with ease, to refine previous convergence results and to uncover new algorithmic variants. In particular, our analysis reveals the vital role played by the step size in textttFL algorithms. The unification also leads to a streamlined and economic way to accelerate textttFL algorithms, without incurring any communication overhead. We perform numerical experiments on both convex and nonconvex models to validate our findings.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 34 of its 63 references that are in the map. First posted on arXiv 12 Aug 2021.
Builds on
34 of 63 references.
References · 34
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2019Advances and Open Problems in Federated Learningcited by 3,021
- 2018Federated Optimization in Heterogeneous Networkscited by 2,971
- 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
- 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
- 2020Adaptive Federated Optimizationcited by 1,036
- 2018LEAF: A Benchmark for Federated Settingscited by 780
- 2018How To Backdoor Federated Learningcited by 745
- 2017Federated Multi-Task Learningcited by 669
- 2020Personalized Federated Learning with Moreau Envelopescited by 612
- 2020Federated Learning with Matched Averagingcited by 564
- 2019Agnostic Federated Learningcited by 451
22 more
- 2019Fair Resource Allocation in Federated Learningcited by 389
- 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
- 2020Adaptive Personalized Federated Learningcited by 353
- 2018Analyzing Federated Learning through an Adversarial Lenscited by 346
- 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
- 2020Three Approaches for Personalization with Applications to Federated Learningcited by 321
- 2020FedML: A Research Library and Benchmark for Federated Machine Learningcited by 298
- 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
- 2019Can You Really Backdoor Federated Learning?cited by 250
- 2020Personalized Federated Learning with First Order Model Optimizationcited by 201
- 2020FedSplit: An algorithmic framework for fast federated optimizationcited by 108
- 2020Robust Federated Learning: The Case of Affine Distribution Shiftscited by 102
- 2020Acceleration for Compressed Gradient Descent in Distributed and Federated Optimizationcited by 90
- 2020Local SGD: Unified Theory and New Efficient Methodscited by 88
- 2019Generative Models for Effective ML on Private, Decentralized Datasetscited by 85
- 2019First Analysis of Local GD on Heterogeneous Datacited by 80
- 2020From Local SGD to Local Fixed-Point Methods for Federated Learningcited by 74
- 2021Convergence and Accuracy Trade-Offs in Federated Learning and Meta-Learningcited by 52
- 2020Faster On-Device Training Using New Federated Momentum Algorithmcited by 25
- 2018Globally Convergent Type-I Anderson Acceleration for Non-Smooth Fixed-Point Iterationscited by 24
- 2017Douglas-Rachford splitting and ADMM for nonconvex optimization: tight convergence resultscited by 21
- 2019Anderson Accelerated Douglas-Rachford Splittingcited by 8
Built on by
Citing papers per year: 2021: 1; 2022: 0; 2023: 0; 2024: 1.
2 papers.