# FedHiSyn: A Hierarchical Synchronous Federated Learning Framework for Resource and Data Heterogeneity

arXiv: https://arxiv.org/abs/2206.10546 · HTML: https://mlsanity.com/paper/2206.10546/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 34 references that are in the map.
- First posted on arXiv 21 Jun 2022.

## Builds on (most cited first, 22 of 22)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Federated Learning with Non-IID Data](https://mlsanity.com/paper/1806.00582/) (2018)
- [Adaptive Federated Optimization](https://mlsanity.com/paper/2003.00295/) (2020)
- [Federated Multi-Task Learning](https://mlsanity.com/paper/1705.10467/) (2017)
- [Federated Learning for Mobile Keyboard Prediction](https://mlsanity.com/paper/1811.03604/) (2018)
- [Federated Learning on Non-IID Data Silos: An Experimental Study](https://mlsanity.com/paper/2102.02079/) (2021)
- [Federated Learning Based on Dynamic Regularization](https://mlsanity.com/paper/2111.04263/) (2021)
- [Exploiting Shared Representations for Personalized Federated Learning](https://mlsanity.com/paper/2102.07078/) (2021)
- [FedMD: Heterogenous Federated Learning via Model Distillation](https://mlsanity.com/paper/1910.03581/) (2019)
- [Fair Resource Allocation in Federated Learning](https://mlsanity.com/paper/1905.10497/) (2019)
- [Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge](https://mlsanity.com/paper/1804.08333/) (2018)
- [Asynchronous Federated Optimization](https://mlsanity.com/paper/1903.03934/) (2019)
- [Communication-Efficient Federated Deep Learning with Asynchronous Model Update and Temporally Weighted Aggregation](https://mlsanity.com/paper/1903.07424/) (2019)
- [From Distributed Machine Learning to Federated Learning: A Survey](https://mlsanity.com/paper/2104.14362/) (2021)
- [FedAT: A High-Performance and Communication-Efficient Federated Learning System with Asynchronous Tiers](https://mlsanity.com/paper/2010.05958/) (2020)
- [IBM Federated Learning: an Enterprise Framework White Paper V0.1](https://mlsanity.com/paper/2007.10987/) (2020)
- [Device Scheduling with Fast Convergence for Wireless Federated Learning](https://mlsanity.com/paper/1911.00856/) (2019)
- [Efficient Device Scheduling with Multi-Job Federated Learning](https://mlsanity.com/paper/2112.05928/) (2021)
- [FedDUAP: Federated Learning with Dynamic Update and Adaptive Pruning Using Shared Data on the Server](https://mlsanity.com/paper/2204.11536/) (2022)
- [CSAFL: A Clustered Semi-Asynchronous Federated Learning Framework](https://mlsanity.com/paper/2104.08184/) (2021)
- [Towards Efficient and Stable K-Asynchronous Federated Learning with Unbounded Stale Gradients on Non-IID Data](https://mlsanity.com/paper/2203.01214/) (2022)

## Built on by (most cited first, 20 of 20)

- [Federated Learning of Large Language Models with Parameter-Efficient Prompt Tuning and Adaptive Optimization](https://mlsanity.com/paper/2310.15080/) (2023)
- [Multi-Job Intelligent Scheduling with Cross-Device Federated Learning](https://mlsanity.com/paper/2211.13430/) (2022)
- [Subspace based Federated Unlearning](https://mlsanity.com/paper/2302.12448/) (2023)
- [Visual Prompt Based Personalized Federated Learning](https://mlsanity.com/paper/2303.08678/) (2023)
- [AEDFL: Efficient Asynchronous Decentralized Federated Learning with Heterogeneous Devices](https://mlsanity.com/paper/2312.10935/) (2023)
- [Trustworthy Federated Learning: Privacy, Security, and Beyond](https://mlsanity.com/paper/2411.01583/) (2024)
- [Efficient Asynchronous Federated Learning with Sparsification and Quantization](https://mlsanity.com/paper/2312.15186/) (2023)
- [Enhancing Trust and Privacy in Distributed Networks: A Comprehensive Survey on Blockchain-based Federated Learning](https://mlsanity.com/paper/2403.19178/) (2024)
- [Quantum Federated Learning With Quantum Networks](https://mlsanity.com/paper/2310.15084/) (2023)
- [A Comprehensive Survey of Federated Transfer Learning: Challenges, Methods and Applications](https://mlsanity.com/paper/2403.01387/) (2024)
- [Efficient Federated Learning Using Dynamic Update and Adaptive Pruning with Momentum on Shared Server Data](https://mlsanity.com/paper/2408.05678/) (2024)
- [Fisher Information-based Efficient Curriculum Federated Learning with Large Language Models](https://mlsanity.com/paper/2410.00131/) (2024)
- [Large-scale Knowledge Distillation with Elastic Heterogeneous Computing Resources](https://mlsanity.com/paper/2207.06667/) (2022)
- [On Mask-based Image Set Desensitization with Recognition Support](https://mlsanity.com/paper/2312.08975/) (2023)
- [Efficient Federated Learning with Timely Update Dissemination](https://mlsanity.com/paper/2507.06031/) (2025)
- [Performance Analysis of Decentralized Federated Learning Deployments](https://mlsanity.com/paper/2503.11828/) (2025)
- [Knowledge Rumination for Client Utility Evaluation in Heterogeneous Federated Learning](https://mlsanity.com/paper/2312.10425/) (2023)
- [SEAFL: Enhancing Efficiency in Semi-Asynchronous Federated Learning through Adaptive Aggregation and Selective Training](https://mlsanity.com/paper/2503.05755/) (2025)
- [DAG-AFL:Directed Acyclic Graph-based Asynchronous Federated Learning](https://mlsanity.com/paper/2507.20571/) (2025)
- [HFedATM: Hierarchical Federated Domain Generalization via Optimal Transport and Regularized Mean Aggregation](https://mlsanity.com/paper/2508.05135/) (2025)

## Abstract (arXiv)

Federated Learning (FL) enables training a global model without sharing the decentralized raw data stored on multiple devices to protect data privacy. Due to the diverse capacity of the devices, FL frameworks struggle to tackle the problems of straggler effects and outdated models. In addition, the data heterogeneity incurs severe accuracy degradation of the global model in the FL training process. To address aforementioned issues, we propose a hierarchical synchronous FL framework, i.e., FedHiSyn. FedHiSyn first clusters all available devices into a small number of categories based on their computing capacity. After a certain interval of local training, the models trained in different categories are simultaneously uploaded to a central server. Within a single category, the devices communicate the local updated model weights to each other based on a ring topology. As the efficiency of training in the ring topology prefers devices with homogeneous resources, the classification based on the computing capacity mitigates the impact of straggler effects. Besides, the combination of the synchronous update of multiple categories and the device communication within a single category help address the data heterogeneity issue while achieving high accuracy. We evaluate the proposed framework based on MNIST, EMNIST, CIFAR10 and CIFAR100 datasets and diverse heterogeneous settings of devices. Experimental results show that FedHiSyn outperforms six baseline methods, e.g., FedAvg, SCAFFOLD, and FedAT, in terms of training accuracy and efficiency.