FedAH: Aggregated Head for Personalized Federated Learning
Not on the map. Why?
Abstract (arXiv)
Recently, Federated Learning (FL) has gained popularity for its privacy-preserving and collaborative learning capabilities. Personalized Federated Learning (PFL), building upon FL, aims to address the issue of statistical heterogeneity and achieve personalization. Personalized-head-based PFL is a common and effective PFL method that splits the model into a feature extractor and a head, where the feature extractor is collaboratively trained and shared, while the head is locally trained and not shared. However, retaining the head locally, although achieving personalization, prevents the model from learning global knowledge in the head, thus affecting the performance of the personalized model. To solve this problem, we propose a novel PFL method called Federated Learning with Aggregated Head (FedAH), which initializes the head with an Aggregated Head at each iteration. The key feature of FedAH is to perform element-level aggregation between the local model head and the global model head to introduce global information from the global model head. To evaluate the effectiveness of FedAH, we conduct extensive experiments on five benchmark datasets in the fields of computer vision and natural language processing. FedAH outperforms ten state-of-the-art FL methods in terms of test accuracy by 2.87%. Additionally, FedAH maintains its advantage even in scenarios where some clients drop out unexpectedly. Our code is open-accessed at https://github.com/heyuepeng/FedAH.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 24 of its 29 references that are in the map. First posted on arXiv 2 Dec 2024.
Builds on
24 of 29 references.
References · 24
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
- 2019Advances and Open Problems in Federated Learningcited by 3,046
- 2018Federated Optimization in Heterogeneous Networkscited by 2,987
- 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
- 2016Bag of Tricks for Efficient Text Classificationcited by 876
- 2021Model-Contrastive Federated Learningcited by 766
- 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
- 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
- 2020Personalized Federated Learning with Moreau Envelopescited by 618
- 2019Federated Learning with Personalization Layerscited by 541
- 2021Exploiting Shared Representations for Personalized Federated Learningcited by 541
12 more
- 2021Towards Personalized Federated Learningcited by 464
- 2019Federated Machine Learning: Concept and Applicationscited by 412
- 2017A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasetscited by 377
- 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 283
- 2021Federated Learning for Internet of Things: A Comprehensive Surveycited by 252
- 2021Personalized Federated Learning using Hypernetworkscited by 212
- 2020Personalized Federated Learning with First Order Model Optimizationcited by 201
- 2018Federated Meta-Learning with Fast Convergence and Efficient Communicationcited by 155
- 2022FedALA: Adaptive Local Aggregation for Personalized Federated Learningcited by 122
- 2021Adapt to Adaptation: Learning Personalization for Cross-Silo Federated Learningcited by 53
- 2023Eliminating Domain Bias for Federated Learning in Representation Spacecited by 38
- 2023FedCP: Separating Feature Information for Personalized Federated Learning via Conditional Policycited by 36
Built on by
Citing papers per year: 2025: 2.
2 papers.