mlsanityData to 7 Oct 2026

On ADMM in Heterogeneous Federated Learning: Personalization, Robustness, and Fairness

arXiv 2407.16397 ↗First posted 23 Jul 2024cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Statistical heterogeneity is a root cause of tension among accuracy, fairness, and robustness of federated learning (FL), and is key in paving a path forward. Personalized FL (PFL) is an approach that aims to reduce the impact of statistical heterogeneity by developing personalized models for individual users, while also inherently providing benefits in terms of fairness and robustness. However, existing PFL frameworks focus on improving the performance of personalized models while neglecting the global model. Moreover, these frameworks achieve sublinear convergence rates and rely on strong assumptions. In this paper, we propose FLAME, an optimization framework by utilizing the alternating direction method of multipliers (ADMM) to train personalized and global models. We propose a model selection strategy to improve performance in situations where clients have different types of heterogeneous data. Our theoretical analysis establishes the global convergence and two kinds of convergence rates for FLAME under mild assumptions. We theoretically demonstrate that FLAME is more robust and fair than the state-of-the-art methods on a class of linear problems. Our experimental findings show that FLAME outperforms state-of-the-art methods in convergence and accuracy, and it achieves higher test accuracy under various attacks and performs more uniformly across clients.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 71 of its 118 references that are in the map. First posted on arXiv 23 Jul 2024.

Builds on

71 of 118 references.

References · 71

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  4. 2019Advances and Open Problems in Federated Learningcited by 3,021
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  6. 2016Equality of Opportunity in Supervised Learningcited by 2,000
  7. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  8. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  9. 2018Federated Learning with Non-IID Datacited by 1,099
  10. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  11. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  12. 2018On First-Order Meta-Learning Algorithmscited by 1,003
38 more
  1. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  2. 2017A Survey on Multi-Task Learningcited by 838
  3. 2018LEAF: A Benchmark for Federated Settingscited by 780
  4. 2018How To Backdoor Federated Learningcited by 745
  5. 2018Byzantine-Robust Distributed Learning: Towards Optimal Statistical Ratescited by 698
  6. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  7. 2017Federated Multi-Task Learningcited by 669
  8. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  9. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  10. 2014Convex Optimization: Algorithms and Complexitycited by 569
  11. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
  12. 2018Federated Learning for Mobile Keyboard Predictioncited by 565
  13. 2018Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networkscited by 500
  14. 2019Local Model Poisoning Attacks to Byzantine-Robust Federated Learningcited by 479
  15. 2021Towards Personalized Federated Learningcited by 462
  16. 2018Adaptive Federated Learning in Resource Constrained Edge Computing Systemscited by 408
  17. 2019Fair Resource Allocation in Federated Learningcited by 389
  18. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 380
  19. 2019A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protectioncited by 371
  20. 2018Fairness Without Demographics in Repeated Loss Minimizationcited by 363
  21. 2018Analyzing Federated Learning through an Adversarial Lenscited by 346
  22. 2019Robust Aggregation for Federated Learningcited by 314
  23. 2020Attack of the Tails: Yes, You Really Can Backdoor Federated Learningcited by 307
  24. 2019Can You Really Backdoor Federated Learning?cited by 250
  25. 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
  26. 2023Towards Building the Federated GPT: Federated Instruction Tuningcited by 184
  27. 2021Federated Learning for Smart Healthcare: A Surveycited by 182
  28. 2020Privacy and Robustness in Federated Learning: Attacks and Defensescited by 163
  29. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
  30. 2018RSA: Byzantine-Robust Stochastic Aggregation Methods for Distributed Learning from Heterogeneous Datasetscited by 148
  31. 2023FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learningcited by 146
  32. 2020FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Datacited by 143
  33. 2019On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithmscited by 132
  34. 2020MetaPoison: Practical General-purpose Clean-label Data Poisoningcited by 114
  35. 2021Decentralized Federated Averagingcited by 114
  36. 2022Federated Learning with Label Distribution Skew via Logits Calibrationcited by 108
  37. 2015Global Convergence of ADMM in Nonconvex Nonsmooth Optimizationcited by 98
  38. 2023Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmarkcited by 97

50 most cited of 71 All 71 (JSON)

Built on by

Not cited yet.