mlsanityData to 7 Oct 2026

Federated Learning with Partial Model Personalization

arXiv 2204.03809 ↗First posted 8 Apr 2022cs.LG cs.DC math.OCcited by 114

Not on the map. Why?

Abstract (arXiv)

We consider two federated learning algorithms for training partially personalized models, where the shared and personal parameters are updated either simultaneously or alternately on the devices. Both algorithms have been proposed in the literature, but their convergence properties are not fully understood, especially for the alternating variant. We provide convergence analyses of both algorithms in the general nonconvex setting with partial participation and delineate the regime where one dominates the other. Our experiments on real-world image, text, and speech datasets demonstrate that (a) partial personalization can obtain most of the benefits of full model personalization with a small fraction of personal parameters, and, (b) the alternating update algorithm often outperforms the simultaneous update algorithm by a small but consistent margin.

Key facts. Cited by 114 papers in the map (data through 7 Oct 2026). Builds on 33 of its 61 references that are in the map. First posted on arXiv 8 Apr 2022.

Builds on

33 of 61 references.

References · 33

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  4. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  5. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  6. 2019Advances and Open Problems in Federated Learningcited by 3,021
  7. 2016Deep Learning with Differential Privacycited by 2,682
  8. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  9. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,140
  10. 2020Adaptive Federated Optimizationcited by 1,036
  11. 2017Federated Multi-Task Learningcited by 669
  12. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
21 more
  1. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  2. 2017Learning multiple visual domains with residual adapterscited by 553
  3. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  4. 2019Federated Learning with Personalization Layerscited by 533
  5. 2016Cross-stitch Networks for Multi-task Learningcited by 466
  6. 2020A Unified Theory of Decentralized SGD with Changing Topology and Local Updatescited by 361
  7. 2020Adaptive Personalized Federated Learningcited by 353
  8. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 322
  9. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 321
  10. 2021A Field Guide to Federated Optimizationcited by 312
  11. 2020Google Landmarks Dataset v2 -- A Large-Scale Benchmark for Instance-Level Recognition and Retrievalcited by 305
  12. 2019The Non-IID Data Quagmire of Decentralized Machine Learningcited by 303
  13. 2017EMNIST: an extension of MNIST to handwritten letterscited by 212
  14. 2013On the Convergence of Decentralized Gradient Descentcited by 158
  15. 2020Federated Visual Classification with Real-World Data Distributioncited by 126
  16. 2019End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architecturescited by 120
  17. 2021Distributionally Robust Federated Averagingcited by 104
  18. 2021Federated Reconstruction: Partially Local Federated Learningcited by 78
  19. 2021Personalized Federated Learning: A Unified Framework and Universal Optimization Techniquescited by 34
  20. 2019Federated User Representation Learningcited by 31
  21. 2020Federated Residual Learningcited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 14; 2023: 38; 2024: 33; 2025: 20; 2026: 9.

114 papers.

Sort:

By area:

  1. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
  2. 2022Where to Begin? On the Impact of Pre-Training and Initialization in Federated Learningcited by 71
  3. 2023FedGH: Heterogeneous Federated Learning with Generalized Global Headercited by 56
  4. 2023Learning from models beyond fine-tuningcited by 47
  5. 2022Federated Learning via Inexact ADMMcited by 44
  6. 2023Momentum Benefits Non-IID Federated Learning Simply and Provablycited by 37
  7. 2022FedPop: A Bayesian Approach for Personalised Federated Learningcited by 35
  8. 2023Towards Personalized Federated Learning via Heterogeneous Model Reassemblycited by 31
  9. 2023Towards Federated Foundation Models: Scalable Dataset Pipelines for Group-Structured Learningcited by 29
  10. 2024FedSelect: Personalized Federated Learning with Customized Selection of Parameters for Fine-Tuningcited by 27
  11. 2022FiT: Parameter Efficient Few-shot Transfer Learning for Personalized and Federated Image Classificationcited by 26
  12. 2023Topology-aware Federated Learning in Edge Computing: A Comprehensive Surveycited by 23
38 more
  1. 2023Bold but Cautious: Unlocking the Potential of Personalized Federated Learning through Cautiously Aggressive Collaborationcited by 23
  2. 2023Layer-wise Linear Mode Connectivitycited by 22
  3. 2025LEO-Split: A Semi-Supervised Split Learning Framework over LEO Satellite Networkscited by 22
  4. 2023FedPerfix: Towards Partial Model Personalization of Vision Transformers in Federated Learningcited by 21
  5. 2024Federated Model Heterogeneous Matryoshka Representation Learningcited by 21
  6. 2022lo-fi: distributed fine-tuning without communicationcited by 19
  7. 2023Dual Personalization on Federated Recommendationcited by 19
  8. 2023GPFL: Simultaneously Learning Global and Personalized Feature Information for Personalized Federated Learningcited by 18
  9. 2023Revisiting Personalized Federated Learning: Robustness Against Backdoor Attackscited by 16
  10. 2023A Survey for Federated Learning Evaluations: Goals and Measurescited by 16
  11. 2023PerAda: Parameter-Efficient Federated Learning Personalization with Generalization Guaranteescited by 15
  12. 2024Decoupling General and Personalized Knowledge in Federated Learning via Additive and Low-Rank Decompositioncited by 15
  13. 2023Fusion of Global and Local Knowledge for Personalized Federated Learningcited by 14
  14. 2023FedBEVT: Federated Learning Bird's Eye View Perception Transformer in Road Traffic Systemscited by 14
  15. 2022PGFed: Personalize Each Client's Global Objective for Federated Learningcited by 13
  16. 2023Share Your Representation Only: Guaranteed Improvement of the Privacy-Utility Tradeoff in Federated Learningcited by 13
  17. 2024Advances in Robust Federated Learning: A Survey with Heterogeneity Considerationscited by 12
  18. 2024Modeling Inter-Intra Heterogeneity for Graph Federated Learningcited by 12
  19. 2023FedSSA: Semantic Similarity-based Aggregation for Efficient Model-Heterogeneous Personalized Federated Learningcited by 10
  20. 2024FedLoGe: Joint Local and Generic Federated Learning under Long-tailed Datacited by 10
  21. 2024Efficiently Assemble Normalization Layers and Regularization for Federated Domain Generalizationcited by 10
  22. 2022Straggler-Resilient Personalized Federated Learningcited by 9
  23. 2022Completely Heterogeneous Federated Learningcited by 9
  24. 2023Federated Recommendation with Additive Personalizationcited by 9
  25. 2023ZooPFL: Exploring Black-box Foundation Models for Personalized Federated Learningcited by 9
  26. 2023pFedES: Model Heterogeneous Personalized Federated Learning with Feature Extractor Sharingcited by 9
  27. 2024Decentralized Directed Collaboration for Personalized Federated Learningcited by 9
  28. 2022Efficient Wireless Federated Learning with Partial Model Aggregationcited by 8
  29. 2023Towards More Suitable Personalization in Federated Learning via Decentralized Partial Model Trainingcited by 8
  30. 2023Partially Personalized Federated Learning: Breaking the Curse of Data Heterogeneitycited by 8
  31. 2024Spectral Co-Distillation for Personalized Federated Learningcited by 8
  32. 2024A Comprehensive Survey of Federated Transfer Learning: Challenges, Methods and Applicationscited by 8
  33. 2025Federated Knowledge Distillation for Multi-Model Architectures Lithography Hotspot Detectioncited by 8
  34. 2023FedGKD: Unleashing the Power of Collaboration in Federated Graph Neural Networkscited by 7
  35. 2022Personalized Federated Learning with Communication Compressioncited by 6
  36. 2022Flow: Per-Instance Personalized Federated Learning Through Dynamic Routingcited by 6
  37. 2023Federated Learning of Shareable Bases for Personalization-Friendly Image Classificationcited by 6
  38. 2024Fisher Information-based Efficient Curriculum Federated Learning with Large Language Modelscited by 6

50 most cited of 114 All 114 (JSON)