mlsanityData to 14 Oct 2026

Communication-Efficient Personalized Distributed Learning with Data and Node Heterogeneity

arXiv 2504.17520 ↗First posted 24 Apr 2025cs.LG cs.DC cs.MAcited by 1

Not on the map. Why?

Abstract (arXiv)

To jointly tackle the challenges of data and node heterogeneity in decentralized learning, we propose a distributed strong lottery ticket hypothesis (DSLTH), based on which a communication-efficient personalized learning algorithm is developed. In the proposed method, each local model is represented as the Hadamard product of global real-valued parameters and a personalized binary mask for pruning. The local model is learned by updating and fusing the personalized binary masks while the real-valued parameters are fixed among different agents. To further reduce the complexity of hardware implementation, we incorporate a group sparse regularization term in the loss function, enabling the learned local model to achieve structured sparsity. Then, a binary mask aggregation algorithm is designed by introducing an intermediate aggregation tensor and adding a personalized fine-tuning step in each iteration, which constrains model updates towards the local data distribution. The proposed method effectively leverages the relativity among agents while meeting personalized requirements in heterogeneous node conditions. We also provide a theoretical proof for the DSLTH, establishing it as the foundation of the proposed method. Numerical simulations confirm the validity of the DSLTH and demonstrate the effectiveness of the proposed algorithm.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 31 of its 43 references that are in the map. First posted on arXiv 24 Apr 2025.

Builds on

31 of 43 references.

References · 31

  1. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  2. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  3. 2020Personalized Federated Learning with Moreau Envelopescited by 618
  4. 2020An Efficient Framework for Clustered Federated Learningcited by 511
  5. 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 502
  6. 2018Adaptive Federated Learning in Resource Constrained Edge Computing Systemscited by 409
  7. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 337
  8. 2018Distributed learning of deep neural network over multiple agentscited by 279
  9. 2019What's Hidden in a Randomly Weighted Neural Network?cited by 237
  10. 2019Model Pruning Enables Efficient Federated Learning on Edge Devicescited by 222
  11. 2018Federated Learning via Over-the-Air Computationcited by 219
  12. 2019Deconstructing Lottery Tickets: Zeros, Signs, and the Supermaskcited by 207
19 more
  1. 2020Proving the Lottery Ticket Hypothesis: Pruning is All You Needcited by 201
  2. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  3. 2016Group Sparse Regularization for Deep Neural Networkscited by 89
  4. 2020Optimal Lottery Tickets via SubsetSum: Logarithmic Over-Parameterization is Sufficientcited by 81
  5. 2022DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Trainingcited by 74
  6. 2020LotteryFL: Personalized and Communication-Efficient Federated Learning with Lottery Ticket Hypothesis on Non-IID Datasetscited by 60
  7. 2021Cluster-driven Graph Federated Learning over Multiple Domainscited by 44
  8. 2019Fully Decentralized Joint Learning of Personalized Models and Collaboration Graphscited by 36
  9. 2021FedHM: Efficient Federated Learning for Heterogeneous Models via Low-rank Factorizationcited by 31
  10. 2022Federated Learning of Large Models at the Edge via Principal Sub-Model Trainingcited by 13
  11. 2021Communication-Efficient and Personalized Federated Lottery Ticket Learningcited by 10
  12. 2023Differentially Private Federated Clustering over Non-IID Datacited by 7
  13. 2025Decentralized and Robust Privacy-Preserving Model Using Blockchain-Enabled Federated Deep Learning in Intelligent Enterprisescited by 7
  14. 2023Distributed Learning over Networks with Graph-Attention-Based Personalizationcited by 6
  15. 2020Hierarchical Group Sparse Regularization for Deep Convolutional Neural Networkscited by 5
  16. 2023DePRL: Achieving Linear Convergence Speedup in Personalized Decentralized Learning with Shared Representationscited by 5
  17. 2025A Lightweight and Secure Deep Learning Model for Privacy-Preserving Federated Learning in Intelligent Enterprisescited by 5
  18. 2024Log-Scale Quantization in Distributed First-Order Methods: Gradient-based Learning from Distributed Datacited by 4
  19. 2024Nonlinear Perturbation-based Non-Convex Optimization over Time-Varying Networkscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Semantic-based Distributed Learning for Diverse and Discriminative Representationscited by 0