# Semi-Decentralized Federated Edge Learning for Fast Convergence on Non-IID Data

arXiv: https://arxiv.org/abs/2104.12678 · HTML: https://mlsanity.com/paper/2104.12678/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 27 references that are in the map.
- First posted on arXiv 26 Apr 2021.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Learning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/1908.07873/) (2019)
- [Optimization Methods for Large-Scale Machine Learning](https://mlsanity.com/paper/1606.04838/) (2016)
- [Federated Learning with Matched Averaging](https://mlsanity.com/paper/2002.06440/) (2020)
- [Adaptive Federated Learning in Resource Constrained Edge Computing Systems](https://mlsanity.com/paper/1804.05271/) (2018)
- [Federated Learning in Mobile Edge Networks: A Comprehensive Survey](https://mlsanity.com/paper/1909.11875/) (2019)
- [Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge](https://mlsanity.com/paper/1804.08333/) (2018)
- [The Non-IID Data Quagmire of Decentralized Machine Learning](https://mlsanity.com/paper/1910.00189/) (2019)
- [Client-Edge-Cloud Hierarchical Federated Learning](https://mlsanity.com/paper/1905.06641/) (2019)
- [Federated Learning over Wireless Fading Channels](https://mlsanity.com/paper/1907.09769/) (2019)
- [D2: Decentralized Training over Decentralized Data](https://mlsanity.com/paper/1803.07068/) (2018)
- [Communication-Efficient Edge AI: Algorithms and Systems](https://mlsanity.com/paper/2002.09668/) (2020)
- [Joint Device Scheduling and Resource Allocation for Latency Constrained Wireless Federated Learning](https://mlsanity.com/paper/2007.07174/) (2020)
- [Multi-Level Local SGD for Heterogeneous Hierarchical Networks](https://mlsanity.com/paper/2007.13819/) (2020)

## Built on by (most cited first, 12 of 12)

- [A Survey of What to Share in Federated Learning: Perspectives on Model Utility, Privacy Leakage, and Communication Efficiency](https://mlsanity.com/paper/2307.10655/) (2023)
- [Scalable and Low-Latency Federated Learning with Cooperative Mobile Edge Networking](https://mlsanity.com/paper/2205.13054/) (2022)
- [HiFlash: Communication-Efficient Hierarchical Federated Learning with Adaptive Staleness Control and Heterogeneity-aware Client-Edge Association](https://mlsanity.com/paper/2301.06447/) (2023)
- [Semi-Decentralized Federated Edge Learning with Data and Device Heterogeneity](https://mlsanity.com/paper/2112.10313/) (2021)
- [Advancements in Federated Learning: Models, Methods, and Privacy](https://mlsanity.com/paper/2302.11466/) (2023)
- [Asynchronous Semi-Decentralized Federated Edge Learning for Heterogeneous Clients](https://mlsanity.com/paper/2112.04737/) (2021)
- [How to Collaborate: Towards Maximizing the Generalization Performance in Cross-Silo Federated Learning](https://mlsanity.com/paper/2401.13236/) (2024)
- [Heterogeneity-Aware Cooperative Federated Edge Learning with Adaptive Computation and Communication Compression](https://mlsanity.com/paper/2409.04022/) (2024)
- [A Comprehensive Survey on Joint Resource Allocation Strategies in Federated Edge Learning](https://mlsanity.com/paper/2410.07881/) (2024)
- [Heterogeneity-Aware Resource Allocation and Topology Design for Hierarchical Federated Edge Learning](https://mlsanity.com/paper/2409.19509/) (2024)
- [FedAvgen: Metadata for Model Aggregation In Communication Systems](https://mlsanity.com/paper/2505.05486/) (2025)
- [Towards Serverless Semi-Decentralized Federated Learning with Heterogeneous Optimizers](https://mlsanity.com/paper/2606.06687/) (2026)

## Abstract (arXiv)

Federated edge learning (FEEL) has emerged as an effective approach to reduce the large communication latency in Cloud-based machine learning solutions, while preserving data privacy. Unfortunately, the learning performance of FEEL may be compromised due to limited training data in a single edge cluster. In this paper, we investigate a novel framework of FEEL, namely semi-decentralized federated edge learning (SD-FEEL). By allowing model aggregation across different edge clusters, SD-FEEL enjoys the benefit of FEEL in reducing the training latency, while improving the learning performance by accessing richer training data from multiple edge clusters. A training algorithm for SD-FEEL with three main procedures in each round is presented, including local model updates, intra-cluster and inter-cluster model aggregations, which is proved to converge on non-independent and identically distributed (non-IID) data. We also characterize the interplay between the network topology of the edge servers and the communication overhead of inter-cluster model aggregation on the training performance. Experiment results corroborate our analysis and demonstrate the effectiveness of SD-FFEL in achieving faster convergence than traditional federated learning architectures. Besides, guidelines on choosing critical hyper-parameters of the training algorithm are also provided.