Federated Learning over Hierarchical Wireless Networks: Training Latency Minimization via Submodel Partitioning
Not on the map. Why?
Abstract (arXiv)
Hierarchical federated learning (HFL) has demonstrated promising scalability advantages over the traditional "star-topology" architecture-based federated learning (FL). However, HFL still imposes significant computation, communication, and storage burdens on the edge, especially when training a large-scale model over resource-constrained wireless devices. In this paper, we propose hierarchical independent submodel training (HIST), a new FL methodology that aims to address these issues in hierarchical cloud-edge-client networks. The key idea behind HIST is to divide the global model into disjoint partitions (or submodels) per round so that each group of clients (i.e., cells) is responsible for training only one partition of the model. We characterize the convergence behavior of HIST under mild assumptions, showing the impacts of several key attributes (e.g., submodel sizes, number of cells, edge and global aggregation frequencies) on the rate and stationarity gap. Building upon the theoretical results, we propose a submodel partitioning strategy to minimize the training latency depending on network resource availability and a target learning performance guarantee. We then demonstrate how HIST can be augmented with over-the-air computation (AirComp) to further enhance the efficiency of the model aggregation over the edge cells. Through numerical evaluations, we verify that HIST is able to save training time and communication costs by wide margins while achieving comparable accuracy as conventional HFL. Moreover, our experiments demonstrate that AirComp-assisted HIST provides further improvements in training latency.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 40 of its 52 references that are in the map. First posted on arXiv 27 Oct 2023.
Builds on
40 of 52 references.
References · 40
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2018Federated Optimization in Heterogeneous Networkscited by 2,971
- 2020An Efficient Framework for Clustered Federated Learningcited by 505
- 2018Adaptive Federated Learning in Resource Constrained Edge Computing Systemscited by 408
- 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
- 2018Federated Learning via Over-the-Air Computationcited by 215
- 2019Client-Edge-Cloud Hierarchical Federated Learningcited by 202
- 2018Broadband Analog Aggregation for Low-Latency Federated Edge Learning (Extended Version)cited by 186
- 2020One-Bit Over-the-Air Aggregation for Communication-Efficient Federated Edge Learning: Design and Convergence Analysiscited by 108
- 2019Hierarchical Federated Learning Across Heterogeneous Cellular Networkscited by 98
28 more
- 2019Federated Learning over Wireless Networks: Convergence Analysis and Resource Allocationcited by 93
- 2023Decentralized Federated Learning: A Survey and Perspectivecited by 86
- 2020Over-the-Air Federated Learning from Heterogeneous Datacited by 72
- 2022Communication-Efficient Stochastic Zeroth-Order Optimization for Federated Learningcited by 59
- 2021Semi-Decentralized Federated Learning with Cooperative D2D Local Model Aggregationscited by 50
- 2021Hierarchical Federated Learning with Quantization: Convergence Analysis and System Designcited by 47
- 2021Optimized Power Control Design for Over-the-Air Federated Edge Learningcited by 45
- 2020Multi-Stage Hybrid Federated Learning over Large-Scale D2D-Enabled Fog Networkscited by 42
- 2021Mobility-Aware Cluster Federated Learning in Hierarchical Wireless Networkscited by 35
- 2022Variance Reduced ProxSkip: Algorithm, Theory and Application to Federated Learningcited by 33
- 2020Reconfigurable Intelligent Surface Enabled Federated Learning: A Unified Communication-Learning Design Approachcited by 32
- 2020Network-Aware Optimization of Distributed Learning for Fog Computingcited by 27
- 2020Demystifying Why Local Aggregation Helps: Convergence Analysis of Hierarchical SGDcited by 26
- 2023Every Parameter Matters: Ensuring the Convergence of Federated Learning with Dynamic Heterogeneous Models Reductioncited by 24
- 2019Distributed Learning of Deep Neural Networks using Independent Subnet Trainingcited by 20
- 2022SplitGP: Achieving Both Generalization and Personalization in Federated Learningcited by 20
- 2021ResIST: Layer-Wise Decomposition of ResNets for Distributed Trainingcited by 18
- 2021Masked Training of Neural Networks with Partial Gradientscited by 15
- 2022Interference Management for Over-the-Air Federated Learning in Multi-Cell Wireless Networkscited by 15
- 2021Over-the-Air Computation via Reconfigurable Intelligent Surfacecited by 12
- 2022Over-the-Air Computation for 6G: Foundations, Technologies, and Applicationscited by 12
- 2019Gradient Descent with Compressed Iteratescited by 11
- 2021GIST: Distributed Training for Large-Scale Graph Convolutional Networkscited by 10
- 2022Knowledge-Guided Learning for Transceiver Design in Over-the-Air Federated Learningcited by 10
- 2022Over-the-Air Federated Edge Learning with Hierarchical Clusteringcited by 10
- 2023Hierarchical Federated Learning in Wireless Networks: Pruning Tackles Bandwidth Scarcity and System Heterogeneitycited by 10
- 2023Over-the-Air Federated Learning and Optimizationcited by 10
- 2023Towards a Better Theoretical Understanding of Independent Subnetwork Trainingcited by 6
Built on by
Citing papers per year: 2024: 1.
1 paper.