mlsanityData to 14 Oct 2026

Personalized Federated Learning via ADMM with Moreau Envelope

arXiv 2311.06756 ↗First posted 12 Nov 2023cs.LG cs.DCcited by 0

Not on the map. Why?

Abstract (arXiv)

Personalized federated learning (PFL) is an approach proposed to address the issue of poor convergence on heterogeneous data. However, most existing PFL frameworks require strong assumptions for convergence. In this paper, we propose an alternating direction method of multipliers (ADMM) for training PFL models with Moreau envelope (FLAME), which achieves a sublinear convergence rate, relying on the relatively weak assumption of gradient Lipschitz continuity. Moreover, due to the gradient-free nature of ADMM, FLAME alleviates the need for hyperparameter tuning, particularly in avoiding the adjustment of the learning rate when training the global model. In addition, we propose a biased client selection strategy to expedite the convergence of training of PFL models. Our theoretical analysis establishes the global convergence under both unbiased and biased client selection strategies. Our experiments validate that FLAME, when trained on heterogeneous data, outperforms state-of-the-art methods in terms of model performance. Regarding communication efficiency, it exhibits an average speedup of 3.75x compared to the baselines. Furthermore, experimental results validate that the biased client selection strategy speeds up the convergence of both personalized and global models.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 45 of its 64 references that are in the map. First posted on arXiv 12 Nov 2023.

Builds on

45 of 64 references.

References · 45

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  4. 2019Advances and Open Problems in Federated Learningcited by 3,046
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  6. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
  7. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  8. 2017A Survey on Multi-Task Learningcited by 840
  9. 2017Federated Multi-Task Learningcited by 672
  10. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
  11. 2020Personalized Federated Learning with Moreau Envelopescited by 618
  12. 2014Convex Optimization: Algorithms and Complexitycited by 575
33 more
  1. 2018Federated Learning for Mobile Keyboard Predictioncited by 572
  2. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 568
  3. 2021Towards Personalized Federated Learningcited by 464
  4. 2019A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protectioncited by 374
  5. 2018Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edgecited by 352
  6. 2021Federated Learning on Non-IID Data: A Surveycited by 346
  7. 2021Federated Learning for Smart Healthcare: A Surveycited by 184
  8. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 161
  9. 2023FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learningcited by 149
  10. 2020FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Datacited by 144
  11. 2022Federated Learning with Partial Model Personalizationcited by 114
  12. 2022Federated Learning with Label Distribution Skew via Logits Calibrationcited by 109
  13. 2020Client Selection and Bandwidth Allocation in Wireless Federated Learning Networks: A Long-Term Perspectivecited by 103
  14. 2015Global Convergence of ADMM in Nonconvex Nonsmooth Optimizationcited by 98
  15. 2022Heterogeneous Ensemble Knowledge Transfer for Training Large Models in Federated Learningcited by 81
  16. 2020Convergence of Update Aware Device Scheduling for Federated Learning at the Wireless Edgecited by 71
  17. 2016CoCoA: A General Framework for Communication-Efficient Distributed Optimizationcited by 69
  18. 2022Federated Learning on Heterogeneous and Long-Tailed Data via Classifier Re-Training with Federated Featurescited by 63
  19. 2022Client Selection in Federated Learning: Principles, Challenges, and Opportunitiescited by 62
  20. 2020FedHome: Cloud-Edge based Personalized Federated Learning for In-Home Health Monitoringcited by 61
  21. 2022FederatedScope: A Flexible Federated Learning Platform for Heterogeneitycited by 53
  22. 2022Federated Learning via Inexact ADMMcited by 44
  23. 2020A Better Alternative to Error Feedback for Communication-Efficient Distributed Learningcited by 43
  24. 2023An Introduction to Bi-level Optimization: Foundations and Applications in Signal Processing and Machine Learningcited by 41
  25. 2020Robust Federated Learning with Noisy Labelscited by 40
  26. 2023Generalizable Heterogeneous Federated Cross-Correlation and Instance Similarity Learningcited by 40
  27. 2021Node Selection Toward Faster Convergence for Federated Learning on Non-IID Datacited by 39
  28. 2014Communication-Efficient Distributed Dual Coordinate Ascentcited by 31
  29. 2023Can Public Large Language Models Help Private Cross-device Federated Learning?cited by 30
  30. 2022Disentangled Federated Learning for Tackling Attributes Skew via Invariant Aggregation and Diversity Transferringcited by 28
  31. 2022FedADMM: A Robust Federated Deep Learning Framework with Adaptivity to System Heterogeneitycited by 25
  32. 2015Adding vs. Averaging in Distributed Primal-Dual Optimizationcited by 19
  33. 2022FedGiA: An Efficient Hybrid Algorithm for Federated Learningcited by 8

Built on by

Not cited yet.