Understanding the Statistical Accuracy-Communication Trade-off in Personalized Federated Learning with Minimax Guarantees
Not on the map. Why?
Abstract (arXiv)
Personalized federated learning (PFL) offers a flexible framework for aggregating information across distributed clients with heterogeneous data. This work considers a personalized federated learning setting that simultaneously learns global and local models. While purely local training has no communication cost, collaborative learning among the clients can leverage shared knowledge to improve statistical accuracy, presenting an accuracy-communication trade-off in personalized federated learning. However, the theoretical analysis of how personalization quantitatively influences sample and algorithmic efficiency and their inherent trade-off is largely unexplored. This paper makes a contribution towards filling this gap, by providing a quantitative characterization of the personalization degree on the tradeoff. The results further offers theoretical insights for choosing the personalization degree. As a side contribution, we establish the minimax optimality in terms of statistical accuracy for a widely studied PFL formulation. The theoretical result is validated on both synthetic and real-world datasets and its generalizability is verified in a non-convex setting.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 44 of its 55 references that are in the map. First posted on arXiv 11 Oct 2024.
Builds on
44 of 55 references.
References · 44
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
- 2018Federated Optimization in Heterogeneous Networkscited by 2,987
- 2017Federated Multi-Task Learningcited by 672
- 2020Personalized Federated Learning with Moreau Envelopescited by 618
- 2019Federated Learning with Personalization Layerscited by 541
- 2021Exploiting Shared Representations for Personalized Federated Learningcited by 541
- 2020Adaptive Personalized Federated Learningcited by 358
- 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 325
- 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 269
- 2020Federated Learning of a Mixture of Global and Local Modelscited by 210
- 2019Adaptive Gradient-Based Meta-Learning Methodscited by 183
- 2023Handbook of Convergence Theorems for (Stochastic) Gradient Methodscited by 163
32 more
- 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 161
- 2018Federated Meta-Learning with Fast Convergence and Efficient Communicationcited by 155
- 2022Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challengescited by 138
- 2020Lower Bounds and Optimal Algorithms for Personalized Federated Learningcited by 123
- 2014Long Short-Term Memory Based Recurrent Neural Network Architectures for Large Vocabulary Speech Recognitioncited by 111
- 2024Federated Fine-tuning of Large Language Models under Heterogeneous Tasks and Client Resourcescited by 93
- 2022FedPrompt: Communication-Efficient and Privacy Preserving Prompt Tuning in Federated Learningcited by 91
- 2020Transfer Learning for High-dimensional Linear Regression: Prediction, Estimation, and Minimax Optimalitycited by 90
- 2021Federated Learning: Opportunities and Challengescited by 84
- 2021Federated Reconstruction: Partially Local Federated Learningcited by 79
- 2024FedBiOT: LLM Local Fine-tuning in Federated Learning without Full Modelcited by 74
- 2022Communication-Efficient Adaptive Federated Learningcited by 55
- 2020FedGroup: Efficient Clustered Federated Learning via Decomposed Data-Driven Measurecited by 46
- 2023Efficient Personalized Federated Learning via Sparse Model-Adaptationcited by 38
- 2022Will Bilevel Optimizers Benefit from Loopscited by 36
- 2022Adaptive and Robust Multi-Task Learningcited by 34
- 2022Personalization Improves Privacy-Accuracy Tradeoffs in Federated Learningcited by 23
- 2023FedDiff: Diffusion Model Driven Federated Learning for Multi-Modal and Multi-Clientscited by 23
- 2023FS-Real: Towards Real-World Cross-Device Federated Learningcited by 22
- 2021Targeting Underrepresented Populations in Precision Medicine: A Federated Transfer Learning Approachcited by 20
- 2023Federated Q-Learning: Linear Regret Speedup with Low Communication Costcited by 19
- 2021Minimax Estimation for Personalized Federated Learning: An Alternative between FedAvg and Local Training?cited by 15
- 2023Federated Transfer-Ordered-Personalized Learning for Driver Monitoring Applicationcited by 11
- 2024TransFusion: Covariate-Shift Robust Transfer Learning for High-Dimensional Regressioncited by 11
- 2024The Power of Extrapolation in Federated Learningcited by 10
- 2024Federated Q-Learning with Reference-Advantage Decomposition: Almost Optimal Regret and Logarithmic Communication Costcited by 10
- 2024FADAS: Towards Federated Adaptive Asynchronous Optimizationcited by 10
- 2021Federated Asymptotics: a model to compare federated learning algorithmscited by 8
- 2023Convergence of First-Order Algorithms for Meta-Learning with Moreau Envelopescited by 8
- 2024Personalization in Human-Robot Interaction through Preference-based Action Representation Learningcited by 4
- 2024AdaTrans: Feature-wise and Sample-wise Adaptive Transfer Learning for High-dimensional Regressioncited by 3
- 2024On the Convergence of FedProx with Extrapolation and Inexact Proxcited by 3
Built on by
Citing papers per year: 2025: 1.
1 paper.