mlsanityData to 14 Oct 2026

Clustered Federated Learning based on Nonconvex Pairwise Fusion

arXiv 2211.04218 ↗First posted 8 Nov 2022cs.LG cs.DCcited by 4

Not on the map. Why?

Abstract (arXiv)

This study investigates clustered federated learning (FL), one of the formulations of FL with non-i.i.d. data, where the devices are partitioned into clusters and each cluster optimally fits its data with a localized model. We propose a clustered FL framework that incorporates a nonconvex penalty to pairwise differences of parameters. Without a priori knowledge of the set of devices in each cluster and the number of clusters, this framework can autonomously estimate cluster structures. To implement the proposed framework, we introduce a novel clustered FL method called Fusion Penalized Federated Clustering (FPFC). Building upon the standard alternating direction method of multipliers (ADMM), FPFC can perform partial updates at each communication round and allows parallel computation with variable workload. These strategies significantly reduce the communication cost while ensuring privacy, making it practical for FL. We also propose a new warmup strategy for hyperparameter tuning in FL settings and explore the asynchronous variant of FPFC (asyncFPFC). Theoretical analysis provides convergence guarantees for FPFC with general losses and establishes the statistical convergence rate under a linear model with squared loss. Extensive experiments have demonstrated the superiority of FPFC compared to current methods, including robustness and generalization capability.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 38 of its 70 references that are in the map. First posted on arXiv 8 Nov 2022.

Builds on

38 of 70 references.

References · 38

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  4. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  5. 2019Advances and Open Problems in Federated Learningcited by 3,046
  6. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  7. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
  8. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  9. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  10. 2017Federated Multi-Task Learningcited by 672
  11. 2020Personalized Federated Learning with Moreau Envelopescited by 618
  12. 2018Local SGD Converges Fast and Communicates Littlecited by 605
26 more
  1. 2020An Efficient Framework for Clustered Federated Learningcited by 511
  2. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 503
  3. 2019Agnostic Federated Learningcited by 456
  4. 2020Adaptive Personalized Federated Learningcited by 358
  5. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 325
  6. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 325
  7. 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 283
  8. 2020Federated Learning of a Mixture of Global and Local Modelscited by 210
  9. 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 192
  10. 2019Federated Evaluation of On-device Personalizationcited by 147
  11. 2020Survey of Personalization Techniques for Federated Learningcited by 145
  12. 2020FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Datacited by 144
  13. 2018LAG: Lazily Aggregated Gradient for Communication-Efficient Distributed Learningcited by 96
  14. 2015Network Lasso: Clustering and Optimization in Large Graphscited by 43
  15. 2020Advances in Asynchronous Parallel and Distributed Optimizationcited by 40
  16. 2019Fully Decentralized Joint Learning of Personalized Models and Collaboration Graphscited by 36
  17. 2022Efficient Distribution Similarity Identification in Clustered Federated Learning via Principal Angles Between Client Data Subspacescited by 36
  18. 2022Learning Across Domains and Devices: Style-Driven Source-Free Domain Adaptation in Clustered Federated Learningcited by 35
  19. 2021FedDR -- Randomized Douglas-Rachford Splitting Algorithms for Nonconvex Federated Composite Optimizationcited by 29
  20. 2015Asynchronous stochastic convex optimizationcited by 19
  21. 2021Communication-efficient SGD: From Local SGD to One-Shot Averagingcited by 19
  22. 2021Asynchronous Iterations in Optimization: New Sequence Results and Sharper Algorithmic Guaranteescited by 17
  23. 2015A concave pairwise fusion approach to subgroup analysiscited by 16
  24. 2018Acceleration of Primal-Dual Methods by Preconditioning and Simple Subproblem Procedurescited by 6
  25. 2020Towards Federated Learning With Byzantine-Robust Client Weightingcited by 6
  26. 2021Convergence on a symmetric accelerated stochastic ADMM with larger stepsizescited by 1

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 1; 2025: 2.

4 papers.

  1. 2025A survey on Clustered Federated Learning: Taxonomy, Analysis and Applicationscited by 2
  2. 2023Medical Federated Model with Mixture of Personalized and Sharing Componentscited by 1
  3. 2024Novel clustered federated learning based on local losscited by 0
  4. 2025Smoothing ADMM for Non-convex and Non-smooth Hierarchical Federated Learningcited by 0