# Layer-wised Model Aggregation for Personalized Federated Learning

arXiv: https://arxiv.org/abs/2205.03993 · HTML: https://mlsanity.com/paper/2205.03993/

## Key facts

- Cited by 96 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 May 2022.

## Built on by (most cited first, 30 of 96)

- [Heterogeneous Federated Learning: State-of-the-art and Research Challenges](https://mlsanity.com/paper/2307.10616/) (2023)
- [Efficient Model Personalization in Federated Learning via Client-Specific Prompt Generation](https://mlsanity.com/paper/2308.15367/) (2023)
- [FedTGP: Trainable Global Prototypes with Adaptive-Margin-Enhanced Contrastive Learning for Data and Model Heterogeneity in Federated Learning](https://mlsanity.com/paper/2401.03230/) (2024)
- [LoRA-FAIR: Federated LoRA Fine-Tuning with Aggregation and Initialization Refinement](https://mlsanity.com/paper/2411.14961/) (2024)
- [FedTP: Federated Learning by Transformer Personalization](https://mlsanity.com/paper/2211.01572/) (2022)
- [FedTune: A Deep Dive into Efficient Federated Fine-Tuning with Pre-trained Transformers](https://mlsanity.com/paper/2211.08025/) (2022)
- [L-DAWA: Layer-wise Divergence Aware Weight Aggregation in Federated Self-Supervised Visual Representation Learning](https://mlsanity.com/paper/2307.07393/) (2023)
- [Harmonizing Generalization and Personalization in Federated Prompt Learning](https://mlsanity.com/paper/2405.09771/) (2024)
- [Bold but Cautious: Unlocking the Potential of Personalized Federated Learning through Cautiously Aggressive Collaboration](https://mlsanity.com/paper/2309.11103/) (2023)
- [Layer-wise Linear Mode Connectivity](https://mlsanity.com/paper/2307.06966/) (2023)
- [PeFLL: Personalized Federated Learning by Learning to Learn](https://mlsanity.com/paper/2306.05515/) (2023)
- [FedAWA: Adaptive Optimization of Aggregation Weights in Federated Learning Using Client Vectors](https://mlsanity.com/paper/2503.15842/) (2025)
- [FedHCA2: Towards Hetero-Client Federated Multi-Task Learning](https://mlsanity.com/paper/2311.13250/) (2023)
- [Achieving Byzantine-Resilient Federated Learning via Layer-Adaptive Sparsified Model Aggregation](https://mlsanity.com/paper/2409.01435/) (2024)
- [Hypernetwork-based Personalized Federated Learning for Multi-Institutional CT Imaging](https://mlsanity.com/paper/2206.03709/) (2022)
- [FedA3I: Annotation Quality-Aware Aggregation for Federated Medical Image Segmentation against Heterogeneous Annotation Noise](https://mlsanity.com/paper/2312.12838/) (2023)
- [Federated Online Adaptation for Deep Stereo](https://mlsanity.com/paper/2405.14873/) (2024)
- [Federated Transfer-Ordered-Personalized Learning for Driver Monitoring Application](https://mlsanity.com/paper/2301.04829/) (2023)
- [Federated Learning Across Decentralized and Unshared Archives for Remote Sensing Image Classification](https://mlsanity.com/paper/2311.06141/) (2023)
- [pFedSim: Similarity-Aware Model Aggregation Towards Personalized Federated Learning](https://mlsanity.com/paper/2305.15706/) (2023)
- [G2uardFL: Safeguarding Federated Learning Against Backdoor Attacks through Attributed Client Graph Clustering](https://mlsanity.com/paper/2306.04984/) (2023)
- [FedUV: Uniformity and Variance for Heterogeneous Federated Learning](https://mlsanity.com/paper/2402.18372/) (2024)
- [Distributed Federated Learning-Based Deep Learning Model for Privacy MRI Brain Tumor Detection](https://mlsanity.com/paper/2404.10026/) (2024)
- [Tackling Data Heterogeneity in Federated Learning via Loss Decomposition](https://mlsanity.com/paper/2408.12300/) (2024)
- [Learning Across Decentralized Multi-Modal Remote Sensing Archives with Federated Learning](https://mlsanity.com/paper/2306.00792/) (2023)
- [A Survey of Federated Evaluation in Federated Learning](https://mlsanity.com/paper/2305.08070/) (2023)
- [NeFL: Nested Model Scaling for Federated Learning with System Heterogeneous Clients](https://mlsanity.com/paper/2308.07761/) (2023)
- [Fed-CO2: Cooperation of Online and Offline Models for Severe Data Heterogeneity in Federated Learning](https://mlsanity.com/paper/2312.13923/) (2023)
- [Federated Learning with Domain Shift Eraser](https://mlsanity.com/paper/2503.13063/) (2025)
- [FedLWS: Federated Learning with Adaptive Layer-wise Weight Shrinking](https://mlsanity.com/paper/2503.15111/) (2025)

## Abstract (arXiv)

Personalized Federated Learning (pFL) not only can capture the common priors from broad range of distributed data, but also support customized models for heterogeneous clients. Researches over the past few years have applied the weighted aggregation manner to produce personalized models, where the weights are determined by calibrating the distance of the entire model parameters or loss values, and have yet to consider the layer-level impacts to the aggregation process, leading to lagged model convergence and inadequate personalization over non-IID datasets. In this paper, we propose a novel pFL training framework dubbed Layer-wised Personalized Federated learning (pFedLA) that can discern the importance of each layer from different clients, and thus is able to optimize the personalized model aggregation for clients with heterogeneous data. Specifically, we employ a dedicated hypernetwork per client on the server side, which is trained to identify the mutual contribution factors at layer granularity. Meanwhile, a parameterized mechanism is introduced to update the layer-wised aggregation weights to progressively exploit the inter-user similarity and realize accurate model personalization. Extensive experiments are conducted over different models and learning tasks, and we show that the proposed methods achieve significantly higher performance than state-of-the-art pFL methods.