Personalized Federated Learning with Multi-branch Architecture
Not on the map. Why?
Abstract (arXiv)
Federated learning (FL) is a decentralized machine learning technique that enables multiple clients to collaboratively train models without requiring clients to reveal their raw data to each other. Although traditional FL trains a single global model with average performance among clients, statistical data heterogeneity across clients has resulted in the development of personalized FL (PFL), which trains personalized models with good performance on each client's data. A key challenge with PFL is how to facilitate clients with similar data to collaborate more in a situation where each client has data from complex distribution and cannot determine one another's distribution. In this paper, we propose a new PFL method (pFedMB) using multi-branch architecture, which achieves personalization by splitting each layer of a neural network into multiple branches and assigning client-specific weights to each branch. We also design an aggregation method to improve the communication efficiency and the model performance, with which each branch is globally updated with weighted averaging by client-specific weights assigned to the branch. pFedMB is simple but effective in facilitating each client to share knowledge with similar clients by adjusting the weights assigned to each branch. We experimentally show that pFedMB performs better than the state-of-the-art PFL methods using the CIFAR10 and CIFAR100 datasets.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 32 of its 42 references that are in the map. First posted on arXiv 15 Nov 2022.
Builds on
32 of 42 references.
References · 32
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
- 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
- 2019Advances and Open Problems in Federated Learningcited by 3,046
- 2018Federated Optimization in Heterogeneous Networkscited by 2,987
- 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
- 2018Federated Learning with Non-IID Datacited by 1,108
- 2021Model-Contrastive Federated Learningcited by 766
- 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
- 2017Federated Multi-Task Learningcited by 672
- 2020Personalized Federated Learning with Moreau Envelopescited by 618
- 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 596
- 2020Federated Learning with Matched Averagingcited by 564
20 more
- 2019Federated Learning with Personalization Layerscited by 541
- 2020An Efficient Framework for Clustered Federated Learningcited by 511
- 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 503
- 2021Towards Personalized Federated Learningcited by 464
- 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 459
- 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 387
- 2020Adaptive Personalized Federated Learningcited by 358
- 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 325
- 2020Three Approaches for Personalization with Applications to Federated Learningcited by 325
- 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 283
- 2019CondConv: Conditionally Parameterized Convolutions for Efficient Inferencecited by 273
- 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 269
- 2020Federated learning with hierarchical clustering of local updates to improve training on non-IID datacited by 223
- 2021Personalized Federated Learning using Hypernetworkscited by 212
- 2020Federated Learning of a Mixture of Global and Local Modelscited by 210
- 2020Personalized Federated Learning with First Order Model Optimizationcited by 201
- 2019Adaptive Gradient-Based Meta-Learning Methodscited by 183
- 2019Federated Evaluation of On-device Personalizationcited by 147
- 2020Salvaging Federated Learning by Local Adaptationcited by 142
- 2020Triple Wins: Boosting Accuracy, Robustness and Efficiency Together by Enabling Input-Adaptive Inferencecited by 45
Built on by
Citing papers per year: 2024: 1.
1 paper.