mlsanityData to 7 Oct 2026

Lower Bounds and Optimal Algorithms for Personalized Federated Learning

arXiv 2010.02372 ↗First posted 5 Oct 2020cs.LG cs.DC math.OCcited by 122

Not on the map. Why?

Abstract (arXiv)

In this work, we consider the optimization formulation of personalized federated learning recently introduced by Hanzely and Richt'arik (2020) which was shown to give an alternative explanation to the workings of local tt SGD methods. Our first contribution is establishing the first lower bounds for this formulation, for both the communication complexity and the local oracle complexity. Our second contribution is the design of several optimal methods matching these lower bounds in almost all regimes. These are the first provably optimal methods for personalized federated learning. Our optimal methods include an accelerated variant of tt FedProx, and an accelerated variance-reduced version of tt FedAvg/Local tt SGD. We demonstrate the practical superiority of our methods through extensive numerical experiments.

Key facts. Cited by 122 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 29; 2022: 35; 2023: 19; 2024: 17; 2025: 15; 2026: 7.

122 papers.

Sort:

By area:

  1. 2021Model-Contrastive Federated Learningcited by 762
  2. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
  3. 2021A Field Guide to Federated Optimizationcited by 312
  4. 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 189
  5. 2022Federated Learning from Pre-Trained Models: A Contrastive Learning Approachcited by 141
  6. 2022ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally!cited by 134
  7. 2021Parameterized Knowledge Transfer for Personalized Federated Learningcited by 108
  8. 2021FedProc: Prototypical Contrastive Federated Learning on Non-IID datacited by 80
  9. 2021FedCM: Federated Learning with Client-level Momentumcited by 77
  10. 2022Divergence-aware Federated Self-Supervised Learningcited by 71
  11. 2021What Do We Mean by Generalization in Federated Learning?cited by 62
  12. 2021Personalized Federated Learning through Local Memorizationcited by 57
38 more
  1. 2022Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learningcited by 45
  2. 2022pFL-Bench: A Comprehensive Benchmark for Personalized Federated Learningcited by 44
  3. 2023Recent Advances on Federated Learning: A Systematic Surveycited by 40
  4. 2021Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspectivecited by 38
  5. 2022On Privacy and Personalization in Cross-Silo Federated Learningcited by 36
  6. 2022FedPop: A Bayesian Approach for Personalised Federated Learningcited by 35
  7. 2021Personalized Federated Learning: A Unified Framework and Universal Optimization Techniquescited by 34
  8. 2021QuPeD: Quantized Personalization via Distillation with Applications to Federated Learningcited by 32
  9. 2022Motley: Benchmarking Heterogeneity and Personalization in Federated Learningcited by 31
  10. 2021FedDR -- Randomized Douglas-Rachford Splitting Algorithms for Nonconvex Federated Composite Optimizationcited by 29
  11. 2022Federated Adaptive Prompt Tuning for Multi-Domain Collaborative Learningcited by 29
  12. 2021A New Look and Convergence Rate of Federated Multi-Task Learning with Laplacian Regularizationcited by 27
  13. 2022On the Impossible Safety of Large AI Modelscited by 27
  14. 2022Differentially Private Label Protection in Split Learningcited by 25
  15. 2023The Best of Both Worlds: Accurate Global and Personalized Models through Federated Learning with Data-Free Hyper-Knowledge Distillationcited by 25
  16. 2023Federated Temporal Difference Learning with Linear Function Approximation under Environmental Heterogeneitycited by 25
  17. 2021Personalized Federated Learning with Contextualized Generalizationcited by 23
  18. 2022DYNAFED: Tackling Client Data Heterogeneity with Global Dynamicscited by 22
  19. 2023PeFLL: Personalized Federated Learning by Learning to Learncited by 21
  20. 2021Exploiting Heterogeneity in Robust Federated Best-Arm Identificationcited by 20
  21. 2021Auto-FedAvg: Learnable Federated Averaging for Multi-Institutional Medical Image Segmentationcited by 19
  22. 2021Vertical Federated Learning without Revealing Intersection Membershipcited by 19
  23. 2022GradSkip: Communication-Accelerated Local Gradient Methods with Better Computational Complexitycited by 17
  24. 2023Federated Prompt Learning for Weather Foundation Models on Devicescited by 17
  25. 2022An Equivalence Between Data Poisoning and Byzantine Gradient Attackscited by 16
  26. 2022Self-Aware Personalized Federated Learningcited by 16
  27. 2021On Bridging Generic and Personalized Federated Learning for Image Classificationcited by 15
  28. 2023PerAda: Parameter-Efficient Federated Learning Personalization with Generalization Guaranteescited by 15
  29. 2024Personalized Adapter for Large Meteorology Model on Devices: Towards Weather Foundation Modelscited by 15
  30. 2023Benchmarking Algorithms for Federated Domain Generalizationcited by 14
  31. 2021D-Cliques: Compensating for Data Heterogeneity with Topology in Decentralized Federated Learningcited by 12
  32. 2021Private Multi-Task Learning: Formulation and Applications to Federated Learningcited by 11
  33. 2021SPATL: Salient Parameter Aggregation and Transfer Learning for Heterogeneous Clients in Federated Learningcited by 11
  34. 2023Personalized Online Federated Learning with Multiple Kernelscited by 11
  35. 2022Personalized Federated Learning via Convex Clusteringcited by 10
  36. 2024Stable Diffusion-based Data Augmentation for Federated Learning with Non-IID Datacited by 10
  37. 2025Tackling Feature and Sample Heterogeneity in Decentralized Multi-Task Learning: A Sheaf-Theoretic Approachcited by 10
  38. 2025FeDaL: Federated Dataset Learning for General Time Series Foundation Modelscited by 10

50 most cited of 122 All 122 (JSON)