mlsanityData to 14 Oct 2026

DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Training

arXiv 2206.00187 ↗First posted 1 Jun 2022cs.LGcited by 74

Not on the map. Why?

Abstract (arXiv)

Personalized federated learning is proposed to handle the data heterogeneity problem amongst clients by learning dedicated tailored local models for each user. However, existing works are often built in a centralized way, leading to high communication pressure and high vulnerability when a failure or an attack on the central server occurs. In this work, we propose a novel personalized federated learning framework in a decentralized (peer-to-peer) communication protocol named Dis-PFL, which employs personalized sparse masks to customize sparse local models on the edge. To further save the communication and computation cost, we propose a decentralized sparse training technique, which means that each local model in Dis-PFL only maintains a fixed number of active parameters throughout the whole local training and peer-to-peer communication process. Comprehensive experiments demonstrate that Dis-PFL significantly saves the communication bottleneck for the busiest node among all clients and, at the same time, achieves higher model accuracy with less computation cost and communication rounds. Furthermore, we demonstrate that our method can easily adapt to heterogeneous local clients with varying computation complexities and achieves better personalized performances.

Key facts. Cited by 74 papers in the map (data through 14 Oct 2026). Builds on 66 of its 79 references that are in the map. First posted on arXiv 1 Jun 2022.

Builds on

66 of 79 references.

References · 66

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  5. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  6. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  7. 2018Group Normalizationcited by 1,524
  8. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
  9. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 787
  10. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  11. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
  12. 2020Personalized Federated Learning with Moreau Envelopescited by 618
38 more
  1. 2018Rethinking the Value of Network Pruningcited by 599
  2. 2020Federated Learning with Matched Averagingcited by 564
  3. 2021Federated Learning Based on Dynamic Regularizationcited by 562
  4. 2019Federated Learning with Personalization Layerscited by 541
  5. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 541
  6. 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 502
  7. 2019Rigging the Lottery: Making All Tickets Winnerscited by 495
  8. 2020Adaptive Personalized Federated Learningcited by 358
  9. 2018Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edgecited by 352
  10. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 337
  11. 2017Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Sciencecited by 328
  12. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 325
  13. 2020Personalized Federated Learning: A Meta-Learning Approachcited by 312
  14. 2019The Non-IID Data Quagmire of Decentralized Machine Learningcited by 305
  15. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 269
  16. 2017TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learningcited by 256
  17. 2021Federated Learning for Internet of Things: A Comprehensive Surveycited by 252
  18. 2018Privacy Amplification by Subsampling: Tight Analyses via Couplings and Divergencescited by 242
  19. 2019Decentralized Stochastic Optimization and Gossip Algorithms with Compressed Communicationcited by 221
  20. 2021Personalized Federated Learning using Hypernetworkscited by 212
  21. 2020Personalized Federated Learning with First Order Model Optimizationcited by 201
  22. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  23. 2019Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterizationcited by 189
  24. 2019Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computationscited by 181
  25. 2020FedBE: Making Bayesian Model Ensemble Applicable to Federated Learningcited by 176
  26. 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
  27. 2016QSGD: Communication-Efficient SGD via Gradient Quantization and Encodingcited by 147
  28. 2020Federated Learning for Internet of Things: Recent Advances, Taxonomy, and Open Challengescited by 139
  29. 2019An Improved Analysis of Training Over-parameterized Deep Neural Networkscited by 121
  30. 2021Decentralized Federated Averagingcited by 114
  31. 2021Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Trainingcited by 110
  32. 2021Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learningcited by 93
  33. 2019Peer-to-peer Federated Learning on Graphscited by 85
  34. 2022The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Trainingcited by 85
  35. 2021Sparse Training via Boosting Pruning Plasticity with Neuroregenerationcited by 84
  36. 2019Communication-efficient distributed SGD with Sketchingcited by 81
  37. 2021Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Datacited by 69
  38. 2017Collaborative Deep Learning in Fixed Topology Networkscited by 60

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 8; 2023: 17; 2024: 29; 2025: 15; 2026: 5.

74 papers.

Sort:

  1. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 161
  2. 2022Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challengescited by 138
  3. 2022Personalized Subgraph Federated Learningcited by 54
  4. 2023Efficient Model Personalization in Federated Learning via Client-Specific Prompt Generationcited by 54
  5. 2024Byzantine-Robust Decentralized Federated Learningcited by 48
  6. 2023Improving the Model Consistency of Decentralized Federated Learningcited by 45
  7. 2022Topology-aware Generalization of Decentralized SGDcited by 38
  8. 2023Personalized Federated Learning under Mixture of Distributionscited by 37
  9. 2022Sparse Random Networks for Communication-Efficient Federated Learningcited by 32
  10. 2024Enhancing One-Shot Federated Learning Through Data and Ensemble Co-Boostingcited by 27
  11. 2022Meta Knowledge Condensation for Federated Learningcited by 26
  12. 2024FIARSE: Model-Heterogeneous Federated Learning via Importance-Aware Submodel Extractioncited by 21
38 more
  1. 2023Visual Prompt Based Personalized Federated Learningcited by 16
  2. 2022Machine Learning with Confidential Computing: A Systematization of Knowledgecited by 15
  3. 2023AEDFL: Efficient Asynchronous Decentralized Federated Learning with Heterogeneous Devicescited by 15
  4. 2023Fusion of Global and Local Knowledge for Personalized Federated Learningcited by 14
  5. 2024FedLoGe: Joint Local and Generic Federated Learning under Long-tailed Datacited by 10
  6. 2024Decentralized Directed Collaboration for Personalized Federated Learningcited by 9
  7. 2024SpaFL: Communication-Efficient Federated Learning with Sparse Models and Low computational Overheadcited by 9
  8. 2025Hadamard product in deep learning: Introduction, Advances and Challengescited by 9
  9. 2023Towards More Suitable Personalization in Federated Learning via Decentralized Partial Model Trainingcited by 8
  10. 2024Federated Dynamical Low-Rank Training with Global Loss Convergence Guaranteescited by 7
  11. 2022Personalized Federated Learning with Communication Compressioncited by 6
  12. 2023DFedADMM: Dual Constraints Controlled Model Inconsistency for Decentralized Federated Learningcited by 6
  13. 2024Fisher Information-based Efficient Curriculum Federated Learning with Large Language Modelscited by 6
  14. 2023FedABC: Targeting Fair Competition in Personalized Federated Learningcited by 5
  15. 2023Efficient Federated Learning with Enhanced Privacy via Lottery Ticket Pruning in Edge Computingcited by 5
  16. 2023Asymmetrically Decentralized Federated Learningcited by 5
  17. 2023DePRL: Achieving Linear Convergence Speedup in Personalized Decentralized Learning with Shared Representationscited by 5
  18. 2024DRACO: Decentralized Asynchronous Federated Learning over Row-Stochastic Wireless Networkscited by 5
  19. 2024Federated Distillation for Medical Image Classification: Towards Trustworthy Computer-Aided Diagnosiscited by 4
  20. 2024NTK-DFL: Enhancing Decentralized Federated Learning in Heterogeneous Settings via Neural Tangent Kernelcited by 4
  21. 2025PFedDST: Personalized Federated Learning with Decentralized Selection Trainingcited by 4
  22. 2024Decentralized Personalized Federated Learning based on a Conditional Sparse-to-Sparser Schemecited by 3
  23. 2024Personalized One-shot Federated Graph Learning for Heterogeneous Clientscited by 3
  24. 2024Review of Mathematical Optimization in Federated Learningcited by 3
  25. 2022Enhancing Efficiency in Multidevice Federated Learning through Data Selectioncited by 2
  26. 2023Sparse Decentralized Federated Learningcited by 2
  27. 2023Competitive Advantage Attacks to Decentralized Federated Learningcited by 2
  28. 2024Privacy Preserved Blood Glucose Level Cross-Prediction: An Asynchronous Decentralized Federated Learning Approachcited by 2
  29. 2024Smart Sampling: Helping from Friendly Neighbors for Decentralized Federated Learningcited by 2
  30. 2024Boosting the Performance of Decentralized Federated Learning via Catalyst Accelerationcited by 2
  31. 2024Learnable Sparse Customization in Heterogeneous Edge Computingcited by 2
  32. 2025Decentralized Dynamic Cooperation of Personalized Models for Federated Continual Learningcited by 2
  33. 2023Structured Cooperative Learning with Graphical Model Priorscited by 1
  34. 2023Sparse Federated Training of Object Detection in the Internet of Vehiclescited by 1
  35. 2024SSFL: Discovering Sparse Unified Subnetworks at Initialization for Efficient Federated Learningcited by 1
  36. 2024Privacy-Preserved Taxi Demand Prediction System Utilizing Distributed Datacited by 1
  37. 2024Fair Decentralized Learningcited by 1
  38. 2024OledFL: Unleashing the Potential of Decentralized Federated Learning via Opposite Lookahead Enhancementcited by 1

50 most cited of 74 All 74 (JSON)