mlsanityData to 7 Oct 2026

Hierarchical Federated Learning with Momentum Acceleration in Multi-Tier Networks

arXiv 2210.14560 ↗First posted 26 Oct 2022cs.LG cs.DCcited by 6

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose Hierarchical Federated Learning with Momentum Acceleration (HierMo), a three-tier worker-edge-cloud federated learning algorithm that applies momentum for training acceleration. Momentum is calculated and aggregated in the three tiers. We provide convergence analysis for HierMo, showing a convergence rate of O(1/T). In the analysis, we develop a new approach to characterize model aggregation, momentum aggregation, and their interactions. Based on this result, {we prove that HierMo achieves a tighter convergence upper bound compared with HierFAVG without momentum}. We also propose HierOPT, which optimizes the aggregation periods (worker-edge and edge-cloud aggregation periods) to minimize the loss given a limited training time.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 26 of its 59 references that are in the map. First posted on arXiv 26 Oct 2022.

Builds on

26 of 59 references.

References · 26

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2016An overview of gradient descent optimization algorithmscited by 1,021
  4. 2014Convex Optimization: Algorithms and Complexitycited by 569
  5. 2021Towards Personalized Federated Learningcited by 462
  6. 2018Adaptive Federated Learning in Resource Constrained Edge Computing Systemscited by 408
  7. 2019Federated Learning in Mobile Edge Networks: A Comprehensive Surveycited by 407
  8. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
  9. 2020Federated learning with hierarchical clustering of local updates to improve training on non-IID datacited by 222
  10. 2019Client-Edge-Cloud Hierarchical Federated Learningcited by 202
  11. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 192
  12. 2019On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimizationcited by 178
14 more
  1. 2020Mime: Mimicking Centralized Stochastic Algorithms in Federated Learningcited by 152
  2. 2018Fast and Faster Convergence of SGD for Over-Parameterized Models and an Accelerated Perceptroncited by 143
  3. 2019SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentumcited by 121
  4. 2019Hierarchical Federated Learning Across Heterogeneous Cellular Networkscited by 98
  5. 2019Accelerating Federated Learning via Momentum Gradient Descentcited by 75
  6. 2018A Unified Analysis of Stochastic Momentum Methods for Deep Learningcited by 54
  7. 2021Hierarchical Federated Learning with Quantization: Convergence Analysis and System Designcited by 47
  8. 2020On the Convergence of Nesterov's Accelerated Gradient Method in Stochastic Settingscited by 27
  9. 2020Demystifying Why Local Aggregation Helps: Convergence Analysis of Hierarchical SGDcited by 26
  10. 2020FedADC: Accelerated Federated Learning with Drift Controlcited by 26
  11. 2020Faster On-Device Training Using New Federated Momentum Algorithmcited by 25
  12. 2020Federated Learning with Nesterov Accelerated Gradientcited by 11
  13. 2020Multi-Level Local SGD for Heterogeneous Hierarchical Networkscited by 9
  14. 2021Coordinating Momenta for Cross-silo Federated Learningcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 3; 2026: 2.

6 papers.

  1. 2024Fast-Convergent and Communication-Alleviated Heterogeneous Hierarchical Federated Learning in Autonomous Drivingcited by 8
  2. 2025FedRTS: Federated Robust Pruning via Combinatorial Thompson Samplingcited by 4
  3. 2025Recursive Offloading for LLM Serving in Multi-tier Networkscited by 2
  4. 2025Beyond Model Scale Limits: End-Edge-Cloud Federated Learning with Self-Rectified Knowledge Agglomerationcited by 1
  5. 2026A Taxonomy and Resolution Strategy for Client-Level Disagreements in Federated Learningcited by 0
  6. 2026SCOPE-FL: A Strategy-proof Chain-based Optimal pareto efficient Federated Learning Systemcited by 0