mlsanityData to 14 Oct 2026

Fusion of Global and Local Knowledge for Personalized Federated Learning

arXiv 2302.11051 ↗First posted 21 Feb 2023cs.LGcited by 14

Not on the map. Why?

Abstract (arXiv)

Personalized federated learning, as a variant of federated learning, trains customized models for clients using their heterogeneously distributed data. However, it is still inconclusive about how to design personalized models with better representation of shared global knowledge and personalized pattern. To bridge the gap, we in this paper explore personalized models with low-rank and sparse decomposition. Specifically, we employ proper regularization to extract a low-rank global knowledge representation (GKR), so as to distill global knowledge into a compact representation. Subsequently, we employ a sparse component over the obtained GKR to fuse the personalized pattern into the global knowledge. As a solution, we propose a two-stage proximal-based algorithm named Federated learning with mixed Sparse and Low-Rank representation (FedSLR) to efficiently search for the mixed models. Theoretically, under proper assumptions, we show that the GKR trained by FedSLR can at least sub-linearly converge to a stationary point of the regularized problem, and that the sparse component being fused can converge to its stationary point under proper settings. Extensive experiments also demonstrate the superior empirical performance of FedSLR. Moreover, FedSLR reduces the number of parameters, and lowers the down-link communication complexity, which are all desirable for federated learning algorithms. Source code is available in urlhttps://github.com/huangtiansheng/fedslr.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 48 of its 64 references that are in the map. First posted on arXiv 21 Feb 2023.

Builds on

48 of 64 references.

References · 48

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  3. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  4. 2018Federated Learning with Non-IID Datacited by 1,108
  5. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
  6. 2021Model-Contrastive Federated Learningcited by 766
  7. 2018How To Backdoor Federated Learningcited by 765
  8. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
  9. 2021Federated Learning Based on Dynamic Regularizationcited by 562
  10. 2019Federated Learning with Personalization Layerscited by 541
  11. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 541
  12. 2016Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Conditioncited by 539
36 more
  1. 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 536
  2. 2020Adaptive Personalized Federated Learningcited by 358
  3. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 325
  4. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 325
  5. 2019Robust Aggregation for Federated Learningcited by 317
  6. 2020Data Poisoning Attacks Against Federated Learning Systemscited by 275
  7. 2020Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategiescited by 198
  8. 2016Practical Secure Aggregation for Federated Learning on User-Held Datacited by 178
  9. 2020FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Datacited by 144
  10. 2013Mini-batch Stochastic Approximation Methods for Nonconvex Stochastic Composite Optimizationcited by 143
  11. 2020Salvaging Federated Learning by Local Adaptationcited by 142
  12. 2020LDP-Fed: Federated Learning with Local Differential Privacycited by 132
  13. 2021Scatterbrain: Unifying Sparse and Low-rank Attention Approximationcited by 130
  14. 2022Federated Learning with Partial Model Personalizationcited by 114
  15. 2021PPFL: Privacy-preserving Federated Learning with Trusted Execution Environmentscited by 93
  16. 2021CRFL: Certifiably Robust Federated Learning against Backdoor Attackscited by 91
  17. 2021FedCM: Federated Learning with Client-level Momentumcited by 77
  18. 2019Edge Intelligence: The Confluence of Edge Computing and Artificial Intelligencecited by 76
  19. 2021Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Modelscited by 76
  20. 2022DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Trainingcited by 74
  21. 2021LightSecAgg: a Lightweight and Versatile Design for Secure Aggregation in Federated Learningcited by 64
  22. 2020An Efficiency-boosting Client Selection Scheme for Federated Learning with Fairness Guaranteecited by 62
  23. 2020LotteryFL: Personalized and Communication-Efficient Federated Learning with Lottery Ticket Hypothesis on Non-IID Datasetscited by 60
  24. 2020Stochastic Client Selection for Federated Learning with Volatile Clientscited by 39
  25. 2022FedADMM: A Federated Primal-Dual Algorithm Allowing Partial Participationcited by 36
  26. 2018A Simple Proximal Stochastic Gradient Method for Nonsmooth Nonconvex Optimizationcited by 33
  27. 2022Achieving Personalized Federated Learning with Sparse Local Modelscited by 31
  28. 2022FedADMM: A Robust Federated Deep Learning Framework with Adaptivity to System Heterogeneitycited by 25
  29. 2022Robust Weight Perturbation for Adversarial Trainingcited by 25
  30. 2021Proximal Gradient Descent-Ascent: Variable Convergence under KL Geometrycited by 18
  31. 2014Douglas-Rachford splitting for nonconvex optimization with application to nonconvex feasibility problemscited by 15
  32. 2021FLIX: A Simple and Communication-Efficient Alternative to Local Methods in Federated Learningcited by 12
  33. 2015Convergence of multi-block Bregman ADMM for nonconvex composite problemscited by 8
  34. 2023FedABC: Targeting Fair Competition in Personalized Federated Learningcited by 5
  35. 2019Stochastic Proximal Methods for Non-Smooth Non-Convex Constrained Sparse Optimizationcited by 4
  36. 2022Factorized-FL: Agnostic Personalized Federated Learning with Kernel Factorization & Similarity Matchingcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 7; 2025: 2; 2026: 1.

14 papers.

Sort:

By area:

  1. 2024Vaccine: Perturbation-aware Alignment for Large Language Models against Harmful Fine-tuning Attackcited by 115
  2. 2024Lisa: Lazy Safety Alignment for Large Language Models against Harmful Fine-tuning Attackcited by 92
  3. 2024Booster: Tackling Harmful Fine-tuning for Large Language Models via Attenuating Harmful Perturbationcited by 92
  4. 2024Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Surveycited by 90
  5. 2023Understanding How Consistency Works in Federated Learning via Stage-wise Relaxed Initializationcited by 21
  6. 2023Visual Prompt Based Personalized Federated Learningcited by 16
  7. 2023Towards Understanding Generalization and Stability Gaps between Centralized and Decentralized Federated Learningcited by 3
  8. 2024Decentralized Personalized Federated Learning based on a Conditional Sparse-to-Sparser Schemecited by 3
  9. 2023PAGE: Equilibrate Personalization and Generalization in Federated Learningcited by 1
  10. 2025Federated Learning via Meta-Variational Dropoutcited by 1
  11. 2024Convergent Differential Privacy Analysis for General Federated Learningcited by 0
  12. 2024Communication-Efficient Personalized Federal Graph Learning via Low-Rank Decompositioncited by 0
2 more
  1. 2025CEPerFed: Communication-Efficient Personalized Federated Learning for Multi-Pulse MRI Classificationcited by 0
  2. 2026Rethinking the Personalized Relaxed Initialization in the Federated Learning: Consistency and Generalizationcited by 0