mlsanityData to 7 Oct 2026

Decentralized Directed Collaboration for Personalized Federated Learning

arXiv 2405.17876 ↗First posted 28 May 2024cs.LG cs.DC math.OCcited by 9

Not on the map. Why?

Abstract (arXiv)

Personalized Federated Learning (PFL) is proposed to find the greatest personalized models for each client. To avoid the central failure and communication bottleneck in the server-based FL, we concentrate on the Decentralized Personalized Federated Learning (DPFL) that performs distributed model training in a Peer-to-Peer (P2P) manner. Most personalized works in DPFL are based on undirected and symmetric topologies, however, the data, computation and communication resources heterogeneity result in large variances in the personalized models, which lead the undirected aggregation to suboptimal personalized performance and unguaranteed convergence. To address these issues, we propose a directed collaboration DPFL framework by incorporating stochastic gradient push and partial model personalized, called Decentralized Federated Partial Gradient Push (DFedPGP). It personalizes the linear classifier in the modern deep model to customize the local solution and learns a consensus representation in a fully decentralized manner. Clients only share gradients with a subset of neighbors based on the directed and asymmetric topologies, which guarantees flexible choices for resource efficiency and better convergence. Theoretically, we show that the proposed DFedPGP achieves a superior convergence rate of O(frac1√T) in the general non-convex setting, and prove the tighter connectivity among clients will speed up the convergence. The proposed method achieves state-of-the-art (SOTA) accuracy in both data and computation heterogeneity scenarios, demonstrating the efficiency of the directed collaboration and partial gradient push.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 50 of its 73 references that are in the map. First posted on arXiv 28 May 2024.

Builds on

50 of 73 references.

References · 50

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2018Group Normalizationcited by 1,514
  4. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  5. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  6. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  7. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  8. 2019Federated Learning with Personalization Layerscited by 533
  9. 2020An Efficient Framework for Clustered Federated Learningcited by 505
  10. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
  11. 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 496
  12. 2021Towards Personalized Federated Learningcited by 462
38 more
  1. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
  2. 2020Adaptive Personalized Federated Learningcited by 353
  3. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
  4. 2019The Non-IID Data Quagmire of Decentralized Machine Learningcited by 303
  5. 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 282
  6. 2020Personalized Federated Learning with First Order Model Optimizationcited by 201
  7. 2021FedBABU: Towards Enhanced Representation for Federated Image Classificationcited by 180
  8. 2019On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimizationcited by 178
  9. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
  10. 2018Stochastic Gradient Push for Distributed Deep Learningcited by 152
  11. 2016Achieving Geometric Convergence for Distributed Optimization over Time-Varying Graphscited by 149
  12. 2022Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challengescited by 137
  13. 2021Decentralized Federated Averagingcited by 114
  14. 2022Federated Learning with Partial Model Personalizationcited by 114
  15. 2023Personalized Federated Learning with Feature Alignment and Classifier Collaborationcited by 97
  16. 2013Distributed optimization over time-varying directed graphscited by 77
  17. 2022DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Trainingcited by 74
  18. 2021Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Datacited by 69
  19. 2021Blockchain Assisted Decentralized Federated Learning (BLADE-FL): Performance Analysis and Resource Allocationcited by 55
  20. 2023Improving the Model Consistency of Decentralized Federated Learningcited by 45
  21. 2023Efficient Personalized Federated Learning via Sparse Model-Adaptationcited by 38
  22. 2022Topology-aware Generalization of Decentralized SGDcited by 37
  23. 2016Gossip training for deep learningcited by 33
  24. 2014Stochastic Gradient-Push for Strongly Convex Functions on Time-Varying Directed Graphscited by 31
  25. 2018A linear algorithm for optimization over directed graphs with geometric convergencecited by 25
  26. 2020Quantized Decentralized Stochastic Learning over Directed Graphscited by 24
  27. 2023FS-Real: Towards Real-World Cross-Device Federated Learningcited by 22
  28. 2023Personalized Decentralized Federated Learning with Knowledge Distillationcited by 21
  29. 2018Asynchronous Gradient-Pushcited by 20
  30. 2021DeEPCA: Decentralized Exact PCA with Linear Convergence Ratecited by 18
  31. 2022Blockchain-based Federated Learning for Industrial Metaverses: Incentive Scheme with Optimal AoIcited by 16
  32. 2021Decentralized Personalized Federated Learning: Lower Bounds and Optimal Algorithm for All Personalization Modescited by 13
  33. 2022Find Your Friends: Personalized Federated Learning with the Right Collaboratorscited by 10
  34. 2023Towards More Suitable Personalization in Federated Learning via Decentralized Partial Model Trainingcited by 8
  35. 2023DFedADMM: Dual Constraints Controlled Model Inconsistency for Decentralized Federated Learningcited by 6
  36. 2023Asymmetrically Decentralized Federated Learningcited by 5
  37. 2015On the Linear Convergence of Distributed Optimization over Directed Graphscited by 2
  38. 2023Sharper Convergence Guarantees for Federated Learning with Partial Model Personalizationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 3; 2026: 3.

9 papers.

  1. 2024DRACO: Decentralized Asynchronous Federated Learning over Row-Stochastic Wireless Networkscited by 5
  2. 2025PFedDST: Personalized Federated Learning with Decentralized Selection Trainingcited by 4
  3. 2024FedReMa: Improving Personalized Federated Learning via Leveraging the Most Relevant Clientscited by 3
  4. 2024Dataset Distillation-based Hybrid Federated Learning on Non-IID Datacited by 2
  5. 2025Unveiling the Power of Multiple Gossip Steps: A Stability-Based Generalization Analysis in Decentralized Trainingcited by 1
  6. 2025Harmonizing Generalization and Personalization in Ring-topology Decentralized Federated Learningcited by 0
  7. 2026Stability and Generalization of Push-Sum Based Decentralized Optimization over Directed Graphscited by 0
  8. 2026Robust Decentralized Personalized Federated Learning via Prediction-Constrained Neighborhood Collaborationcited by 0
  9. 2026SAGE: Optimal-Stopping Peer Selection for Decentralised Federated Learningcited by 0