mlsanityData to 14 Oct 2026

Personalized Federated Learning: A Unified Framework and Universal Optimization Techniques

arXiv 2102.09743 ↗First posted 19 Feb 2021cs.LGcited by 34

Not on the map. Why?

Abstract (arXiv)

We investigate the optimization aspects of personalized Federated Learning (FL). We propose general optimizers that can be applied to numerous existing personalized FL objectives, specifically a tailored variant of Local SGD and variants of accelerated coordinate descent/accelerated SVRCD. By examining a general personalized objective capable of recovering many existing personalized FL objectives as special cases, we develop a comprehensive optimization theory applicable to a wide range of strongly convex personalized FL models in the literature. We showcase the practicality and/or optimality of our methods in terms of communication and local computation. Remarkably, our general optimization solvers and theory can recover the best-known communication and computation guarantees for addressing specific personalized FL objectives. Consequently, our proposed methods can serve as universal optimizers, rendering the design of task-specific optimizers unnecessary in many instances.

Key facts. Cited by 34 papers in the map (data through 14 Oct 2026). Builds on 46 of its 96 references that are in the map. First posted on arXiv 19 Feb 2021.

Builds on

46 of 96 references.

References · 46

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  4. 2019Advances and Open Problems in Federated Learningcited by 3,046
  5. 2017Federated Multi-Task Learningcited by 672
  6. 2020Personalized Federated Learning with Moreau Envelopescited by 618
  7. 2018Local SGD Converges Fast and Communicates Littlecited by 605
  8. 2018Federated Learning for Mobile Keyboard Predictioncited by 572
  9. 2019Federated Learning with Personalization Layerscited by 541
  10. 2019Meta-Learning with Implicit Gradientscited by 525
  11. 2018Deep Learning for Classical Japanese Literaturecited by 422
  12. 2014SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectivescited by 402
34 more
  1. 2020Adaptive Personalized Federated Learningcited by 358
  2. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 325
  3. 2020Personalized Federated Learning: A Meta-Learning Approachcited by 312
  4. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 269
  5. 2020Federated Learning of a Mixture of Global and Local Modelscited by 210
  6. 2019Adaptive Gradient-Based Meta-Learning Methodscited by 183
  7. 2018Federated Meta-Learning with Fast Convergence and Efficient Communicationcited by 155
  8. 2019On the Convergence of Local Descent Methods in Federated Learningcited by 153
  9. 2020Survey of Personalization Techniques for Federated Learningcited by 145
  10. 2020Lower Bounds and Optimal Algorithms for Personalized Federated Learningcited by 123
  11. 2020Local SGD: Unified Theory and New Efficient Methodscited by 88
  12. 2016Tight Complexity Bounds for Optimizing Composite Objectivescited by 48
  13. 2018Optimal Algorithms for Non-Smooth Distributed Optimization in Networkscited by 41
  14. 2018Graph Oracle Models, Lower Bounds, and Gaps for Parallel Stochastic Optimizationcited by 34
  15. 2015An optimal randomized incremental gradient methodcited by 33
  16. 2015Even Faster Accelerated Coordinate Descent Using Non-Uniform Samplingcited by 23
  17. 2020Federated Residual Learningcited by 23
  18. 2014Coauthorship and Citation Networks for Statisticianscited by 22
  19. 2020An Optimal Algorithm for Decentralized Finite Sum Optimizationcited by 19
  20. 2018Accelerated Coordinate Descent with Arbitrary Sampling and Best Rates for Minibatchescited by 11
  21. 2020Federated Block Coordinate Descent Scheme for Learning Global and Personalized Modelscited by 11
  22. 2020Variance Reduced Coordinate Descent with Acceleration: New Method With a Surprising Application to Finite-Sum Problemscited by 8
  23. 2019Estimating Differential Latent Variable Graphical Models with Applications to Brain Connectivitycited by 7
  24. 2016Sketching Meets Random Projection in the Dual: A Provable Recovery Algorithm for Big and High-dimensional Datacited by 6
  25. 2018Distributed Stochastic Multi-Task Learning with Graph Regularizationcited by 6
  26. 2018High-dimensional Varying Index Coefficient Models via Stein's Identitycited by 6
  27. 2014Optimal variable selection in multi-group sparse discriminant analysiscited by 5
  28. 2015ROCKET: Robust Confidence Intervals via Kendall's Tau for Transelliptical Graphical Modelscited by 5
  29. 2015Kernel Meets Sieve: Post-Regularization Confidence Bands for Sparse Additive Modelcited by 5
  30. 2016Distributed Multi-Task Learning with Shared Representationcited by 5
  31. 2018Recovery of simultaneous low rank and two-way sparse coefficient matrices, a nonconvex approachcited by 5
  32. 2019Simultaneous Inference for Pairwise Graphical Models with Generalized Score Matchingcited by 5
  33. 2018High-dimensional Index Volatility Models via Stein's Identitycited by 3
  34. 2020Semiparametric Nonlinear Bipartite Graph Representation Learning with Provable Guaranteescited by 2

Built on by

Cited by year

Citing papers per year: 2021: 6; 2022: 9; 2023: 11; 2024: 4; 2025: 4.

34 papers.

Sort:

  1. 2021A Field Guide to Federated Optimizationcited by 315
  2. 2022Federated Learning with Partial Model Personalizationcited by 114
  3. 2022DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Trainingcited by 74
  4. 2022Federated Learning via Inexact ADMMcited by 44
  5. 2022FedPop: A Bayesian Approach for Personalised Federated Learningcited by 35
  6. 2021FedCV: A Federated Learning Framework for Diverse Computer Vision Taskscited by 32
  7. 2022Motley: Benchmarking Heterogeneity and Personalization in Federated Learningcited by 31
  8. 2021The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learningcited by 23
  9. 2022Personalization Improves Privacy-Accuracy Tradeoffs in Federated Learningcited by 23
  10. 2023Revisiting Personalized Federated Learning: Robustness Against Backdoor Attackscited by 16
  11. 2023Visual Prompt Based Personalized Federated Learningcited by 16
  12. 2023FedBEVT: Federated Learning Bird's Eye View Perception Transformer in Road Traffic Systemscited by 14
22 more
  1. 2021Linear Speedup in Personalized Collaborative Learningcited by 9
  2. 2021The Minimax Complexity of Distributed Optimizationcited by 7
  3. 2021Adaptive Client Sampling in Federated Learning via Online Learning with Bandit Feedbackcited by 7
  4. 2023Distributed Personalized Empirical Risk Minimizationcited by 6
  5. 2022PersA-FL: Personalized Asynchronous Federated Learningcited by 5
  6. 2023Personalised Federated Learning On Heterogeneous Feature Spacescited by 5
  7. 2023FedABC: Targeting Fair Competition in Personalized Federated Learningcited by 5
  8. 2025Multiplayer Federated Learning: Reaching Equilibrium with Less Communicationcited by 5
  9. 2024Communication-Efficient Adaptive Batch Size Strategies for Distributed Local Gradient Methodscited by 4
  10. 2025PFedDST: Personalized Federated Learning with Decentralized Selection Trainingcited by 4
  11. 2022Personalized Federated Learning with Multiple Known Clusterscited by 3
  12. 2024AdaTrans: Feature-wise and Sample-wise Adaptive Transfer Learning for High-dimensional Regressioncited by 3
  13. 2023Federated attention consistent learning models for prostate cancer diagnosis and Gleason gradingcited by 2
  14. 2024Smart Sampling: Helping from Friendly Neighbors for Decentralized Federated Learningcited by 2
  15. 2024FedPeWS: Personalized Warmup via Subnetworks for Enhanced Heterogeneous Federated Learningcited by 2
  16. 2023PPFL: A Personalized Federated Learning Framework for Heterogeneous Populationcited by 1
  17. 2023Harnessing the Power of Federated Learning in Federated Contextual Banditscited by 1
  18. 2025FedAPM: Federated Learning via ADMM with Partial Model Personalizationcited by 1
  19. 2022Better Methods and Theory for Federated Learning: Compression, Client Selection and Heterogeneitycited by 0
  20. 2023IP-FL: Incentivized and Personalized Federated Learningcited by 0
  21. 2023Federated Deep Equilibrium Learning: Harnessing Compact Global Representations to Enhance Personalizationcited by 0
  22. 2025Personalized and Resilient Distributed Learning Through Opinion Dynamicscited by 0