# FedGradNorm: Personalized Federated Gradient-Normalized Multi-Task Learning

arXiv: https://arxiv.org/abs/2203.13663 · HTML: https://mlsanity.com/paper/2203.13663/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 12 references that are in the map.
- First posted on arXiv 24 Mar 2022.

## Builds on (most cited first, 9 of 9)

- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks](https://mlsanity.com/paper/1711.02257/) (2017)
- [Federated Learning with Personalization Layers](https://mlsanity.com/paper/1912.00818/) (2019)
- [Exploiting Shared Representations for Personalized Federated Learning](https://mlsanity.com/paper/2102.07078/) (2021)
- [Multi-Task Learning with Deep Neural Networks: A Survey](https://mlsanity.com/paper/2009.09796/) (2020)
- [Adaptive Personalized Federated Learning](https://mlsanity.com/paper/2003.13461/) (2020)
- [Bilevel Optimization: Convergence Analysis and Enhanced Design](https://mlsanity.com/paper/2010.07962/) (2020)
- [A Review on Bilevel Optimization: From Classical to Evolutionary Approaches and Applications](https://mlsanity.com/paper/1705.06270/) (2017)
- [Multi-task Learning Approach for Automatic Modulation and Wireless Signal Classification](https://mlsanity.com/paper/2101.10254/) (2021)

## Built on by (most cited first, 9 of 9)

- [FedICT: Federated Multi-task Distillation for Multi-access Edge Computing](https://mlsanity.com/paper/2301.00389/) (2023)
- [Towards Multi-modal Transformers in Federated Learning](https://mlsanity.com/paper/2404.12467/) (2024)
- [PersA-FL: Personalized Asynchronous Federated Learning](https://mlsanity.com/paper/2210.01176/) (2022)
- [Multi-Receiver Task-Oriented Communications via Multi-Task Deep Learning](https://mlsanity.com/paper/2308.06884/) (2023)
- [Personalized Decentralized Multi-Task Learning Over Dynamic Communication Graphs](https://mlsanity.com/paper/2212.11268/) (2022)
- [Hierarchical Over-the-Air FedGradNorm](https://mlsanity.com/paper/2212.07414/) (2022)
- [rAge-k: Communication-Efficient Federated Learning Using Age Factor](https://mlsanity.com/paper/2410.22192/) (2024)
- [AdRo-FL: Informed and Secure Client Selection for Federated Learning in the Presence of Adversarial Aggregator](https://mlsanity.com/paper/2506.17805/) (2025)
- [ColNet: Collaborative Optimization in Decentralized Federated Multi-task Learning Systems](https://mlsanity.com/paper/2501.10347/) (2025)

## Abstract (arXiv)

Multi-task learning (MTL) is a novel framework to learn several tasks simultaneously with a single shared network where each task has its distinct personalized header network for fine-tuning. MTL can be implemented in federated learning settings as well, in which tasks are distributed across clients. In federated settings, the statistical heterogeneity due to different task complexities and data heterogeneity due to non-iid nature of local datasets can both degrade the learning performance of the system. In addition, tasks can negatively affect each other's learning performance due to negative transference effects. To cope with these challenges, we propose FedGradNorm which uses a dynamic-weighting method to normalize gradient norms in order to balance learning speeds among different tasks. FedGradNorm improves the overall learning performance in a personalized federated learning setting. We provide convergence analysis for FedGradNorm by showing that it has an exponential convergence rate. We also conduct experiments on multi-task facial landmark (MTFL) and wireless communication system dataset (RadComDynamic). The experimental results show that our framework can achieve faster training performance compared to equal-weighting strategy. In addition to improving training speed, FedGradNorm also compensates for the imbalanced datasets among clients.