mlsanityData to 14 Oct 2026

PerAda: Parameter-Efficient Federated Learning Personalization with Generalization Guarantees

arXiv 2302.06637 ↗First posted 13 Feb 2023cs.LG cs.AIcited by 15

Not on the map. Why?

Abstract (arXiv)

Personalized Federated Learning (pFL) has emerged as a promising solution to tackle data heterogeneity across clients in FL. However, existing pFL methods either (1) introduce high communication and computation costs or (2) overfit to local data, which can be limited in scope, and are vulnerable to evolved test samples with natural shifts. In this paper, we propose PerAda, a parameter-efficient pFL framework that reduces communication and computational costs and exhibits superior generalization performance, especially under test-time distribution shifts. PerAda reduces the costs by leveraging the power of pretrained models and only updates and communicates a small number of additional parameters from adapters. PerAda has good generalization since it regularizes each client's personalized adapter with a global adapter, while the global adapter uses knowledge distillation to aggregate generalized information from all clients. Theoretically, we provide generalization bounds to explain why PerAda improves generalization, and we prove its convergence to stationary points under non-convex settings. Empirically, PerAda demonstrates competitive personalized performance (+4.85% on CheXpert) and enables better out-of-distribution generalization (+5.23% on CIFAR-10-C) on different datasets across natural and medical domains compared with baselines, while only updating 12.6% of parameters per model based on the adapter. Our code is available at https://github.com/NVlabs/PerAda.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 53 of its 66 references that are in the map. First posted on arXiv 13 Feb 2023.

Builds on

53 of 66 references.

References · 53

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  5. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  6. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  7. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  8. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  9. 2019Advances and Open Problems in Federated Learningcited by 3,046
  10. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  11. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  12. 2016Deep Learning with Differential Privacycited by 2,728
38 more
  1. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  2. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,423
  3. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  4. 2017Deep Hashing Network for Unsupervised Domain Adaptationcited by 1,188
  5. 2020Adaptive Federated Optimizationcited by 1,046
  6. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,026
  7. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
  8. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  9. 2017Federated Multi-Task Learningcited by 672
  10. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
  11. 2020Personalized Federated Learning with Moreau Envelopescited by 618
  12. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 571
  13. 2021Federated Learning Based on Dynamic Regularizationcited by 562
  14. 2017Learning multiple visual domains with residual adapterscited by 557
  15. 2019Federated Learning with Personalization Layerscited by 541
  16. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 541
  17. 2020An Efficient Framework for Clustered Federated Learningcited by 511
  18. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 503
  19. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 459
  20. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 412
  21. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 383
  22. 2020Adaptive Personalized Federated Learningcited by 358
  23. 2020Visual Transformers: Token-based Image Representation and Processing for Computer Visioncited by 334
  24. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 325
  25. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 325
  26. 2020Personalized Federated Learning: A Meta-Learning Approachcited by 312
  27. 2020Federated Learning of a Mixture of Global and Local Modelscited by 210
  28. 2018Do CIFAR-10 Classifiers Generalize to CIFAR-10?cited by 200
  29. 2020FedBE: Making Bayesian Model Ensemble Applicable to Federated Learningcited by 176
  30. 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
  31. 2019Federated Evaluation of On-device Personalizationcited by 147
  32. 2020Salvaging Federated Learning by Local Adaptationcited by 142
  33. 2020Lower Bounds and Optimal Algorithms for Personalized Federated Learningcited by 123
  34. 2022Federated Learning with Partial Model Personalizationcited by 114
  35. 2021Parameterized Knowledge Transfer for Personalized Federated Learningcited by 109
  36. 2021Personalized Federated Learning through Local Memorizationcited by 57
  37. 2022Test-Time Robust Personalization for Federated Learningcited by 39
  38. 2022On Privacy and Personalization in Cross-Silo Federated Learningcited by 36

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 4; 2025: 6; 2026: 2.

15 papers.

Sort:

By area:

  1. 2023FedHCA2: Towards Hetero-Client Federated Multi-Task Learningcited by 17
  2. 2024Personalized Federated Fine-Tuning for LLMs via Data-Driven Heterogeneous Model Architecturescited by 12
  3. 2024SPD-CFL: Stepwise Parameter Dropout for Efficient Continual Federated Learningcited by 5
  4. 2023Towards Open Federated Learning Platforms: Survey and Vision from Technical and Legal Perspectivescited by 3
  5. 2023Joint Local Relational Augmentation and Global Nash Equilibrium for Federated Learning with Non-IID Datacited by 3
  6. 2024FedHPL: Efficient Heterogeneous Federated Learning with Prompt Tuning and Logit Distillationcited by 3
  7. 2025Gradient Inversion Attacks on Parameter-Efficient Fine-Tuningcited by 3
  8. 2024Boosting the Performance of Decentralized Federated Learning via Catalyst Accelerationcited by 2
  9. 2025Federated Adapter on Foundation Models: An Out-Of-Distribution Approachcited by 2
  10. 2025pFedSAM: Personalized Federated Learning of Segment Anything Model for Medical Image Segmentationcited by 1
  11. 2026Personalized Longitudinal Medical Report Generation via Temporally-Aware Federated Adaptationcited by 1
  12. 2025DART: A Server-side Plug-in for Resource-efficient Robust Federated Learningcited by 0
3 more
  1. 2025Cooperative Pseudo Labeling for Unsupervised Federated Classificationcited by 0
  2. 2025FedSDWC: Federated Synergistic Dual-Representation Weak Causal Learning for OODcited by 0
  3. 2026HEART-PFL: Stable Personalized Federated Learning under Heterogeneity with Hierarchical Directional Alignment and Adversarial Knowledge Transfercited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2020PFL-MoE: Personalized Federated Learning Based on Mixture of Experts
  2. 2025Look Back for More: Harnessing Historical Sequential Updates for Personalized Federated Adapter Tuning
  3. 2022Adaptive Expert Models for Personalization in Federated Learning
  4. 2023UPFL: Unsupervised Personalized Federated Learning towards New Clients
  5. 2026Global Federated Learning Strategies for Building Efficient Personalized Models
  6. 2025PM-MOE: Mixture of Experts on Private Model Parameters for Personalized Federated Learning
  7. 2025A Survey on Foundation Models for Personalized Federated Intelligence
  8. 2022A Generative Framework for Personalized Learning and Estimation: Theory, Algorithms, and Privacy