mlsanityData to 7 Oct 2026

Learnable Sparse Customization in Heterogeneous Edge Computing

arXiv 2412.07216 ↗First posted 10 Dec 2024cs.DC cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

To effectively manage and utilize massive distributed data at the network edge, Federated Learning (FL) has emerged as a promising edge computing paradigm across data silos. However, FL still faces two challenges: system heterogeneity (i.e., the diversity of hardware resources across edge devices) and statistical heterogeneity (i.e., non-IID data). Although sparsification can extract diverse submodels for diverse clients, most sparse FL works either simply assign submodels with artificially-given rigid rules or prune partial parameters using heuristic strategies, resulting in inflexible sparsification and poor performance. In this work, we propose Learnable Personalized Sparsification for heterogeneous Federated learning (FedLPS), which achieves the learnable customization of heterogeneous sparse models with importance-associated patterns and adaptive ratios to simultaneously tackle system and statistical heterogeneity. Specifically, FedLPS learns the importance of model units on local data representation and further derives an importance-based sparse pattern with minimal heuristics to accurately extract personalized data features in non-IID settings. Furthermore, Prompt Upper Confidence Bound Variance (P-UCBV) is designed to adaptively determine sparse ratios by learning the superimposed effect of diverse device capabilities and non-IID data, aiming at resource self-adaptation with promising accuracy. Extensive experiments show that FedLPS outperforms status quo approaches in accuracy and training costs, which improves accuracy by 1.28%-59.34% while reducing running time by more than 68.80%.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 54 of its 80 references that are in the map. First posted on arXiv 10 Dec 2024.

Builds on

54 of 80 references.

References · 54

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2019Advances and Open Problems in Federated Learningcited by 3,021
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  5. 2018LEAF: A Benchmark for Federated Settingscited by 780
  6. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  7. 2018Rethinking the Value of Network Pruningcited by 595
  8. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  9. 2016Learning Structured Sparsity in Deep Neural Networkscited by 538
  10. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  11. 2019Federated Learning with Personalization Layerscited by 533
  12. 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 528
38 more
  1. 2019Rigging the Lottery: Making All Tickets Winnerscited by 491
  2. 2021Towards Personalized Federated Learningcited by 462
  3. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
  4. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 380
  5. 2019A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protectioncited by 371
  6. 2018Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edgecited by 347
  7. 2021Federated Learning on Non-IID Data: A Surveycited by 345
  8. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
  9. 2019The Non-IID Data Quagmire of Decentralized Machine Learningcited by 303
  10. 2019Model Pruning Enables Efficient Federated Learning on Edge Devicescited by 219
  11. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  12. 2018Expanding the Reach of Federated Learning by Reducing Client Resource Requirementscited by 162
  13. 2020Soft Threshold Weight Reparameterization for Learnable Sparsitycited by 150
  14. 2022FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extractioncited by 102
  15. 2017MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networkscited by 92
  16. 2021FedCM: Federated Learning with Client-level Momentumcited by 77
  17. 2018AI Benchmark: Running Deep Neural Networks on Android Smartphonescited by 75
  18. 2022DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Trainingcited by 74
  19. 2023Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformerscited by 59
  20. 2022Self-supervised Trajectory Representation Learning with Temporal Regularities and Travel Semanticscited by 58
  21. 2021Federated Dropout -- A Simple Approach for Enabling Federated Learning on Resource Constrained Devicescited by 56
  22. 2022Advancing Model Pruning via Bi-level Optimizationcited by 55
  23. 2021Federated Dynamic Sparse Training: Computing Less, Communicating Less, Yet Learning Bettercited by 54
  24. 2023Recent Advances on Federated Learning: A Systematic Surveycited by 40
  25. 2020Adaptive Federated Dropout: Improving Communication Efficiency and Generalization for Federated Learningcited by 37
  26. 2022No One Left Behind: Inclusive Federated Learning over Heterogeneous Devicescited by 36
  27. 2023FedCP: Separating Feature Information for Personalized Federated Learning via Conditional Policycited by 36
  28. 2022Achieving Personalized Federated Learning with Sparse Local Modelscited by 30
  29. 2022A Multi-agent Reinforcement Learning Approach for Efficient Client Selection in Federated Learningcited by 28
  30. 2021Resource-Efficient Federated Learningcited by 26
  31. 2022FedADMM: A Robust Federated Deep Learning Framework with Adaptivity to System Heterogeneitycited by 25
  32. 2022FedCross: Towards Accurate Federated Learning via Multi-Model Cross-Aggregationcited by 21
  33. 2017Efficient-UCBV: An Almost Optimal Algorithm using Variance Estimatescited by 20
  34. 2024FedP3: Federated Personalized and Privacy-friendly Network Pruning under Model Heterogeneitycited by 13
  35. 2023Complement Sparsification: Low-Overhead Model Pruning for Federated Learningcited by 12
  36. 2023AdaptiveFL: Adaptive Heterogeneous Federated Learning for Resource-Constrained AIoT Systemscited by 9
  37. 2024Full Bayesian Significance Testing for Neural Networkscited by 8
  38. 2023FedTrip: A Resource-Efficient Federated Learning Method with Triplet Regularizationcited by 6

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025RoSA: Enhancing Parameter-Efficient Fine-Tuning via RoPE-aware Selective Adaptation in Large Language Modelscited by 0
  2. 2026Capacity Confounds and Coverage Guarantees in Adaptive Sub-model Federated Learningcited by 0