# Hierarchical Federated Learning with Momentum Acceleration in Multi-Tier Networks

arXiv: https://arxiv.org/abs/2210.14560 · HTML: https://mlsanity.com/paper/2210.14560/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 59 references that are in the map.
- First posted on arXiv 26 Oct 2022.

## Builds on (most cited first, 26 of 26)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [An overview of gradient descent optimization algorithms](https://mlsanity.com/paper/1609.04747/) (2016)
- [Convex Optimization: Algorithms and Complexity](https://mlsanity.com/paper/1405.4980/) (2014)
- [Towards Personalized Federated Learning](https://mlsanity.com/paper/2103.00710/) (2021)
- [Adaptive Federated Learning in Resource Constrained Edge Computing Systems](https://mlsanity.com/paper/1804.05271/) (2018)
- [Federated Learning in Mobile Edge Networks: A Comprehensive Survey](https://mlsanity.com/paper/1909.11875/) (2019)
- [FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization](https://mlsanity.com/paper/1909.13014/) (2019)
- [Federated learning with hierarchical clustering of local updates to improve training on non-IID data](https://mlsanity.com/paper/2004.11791/) (2020)
- [Client-Edge-Cloud Hierarchical Federated Learning](https://mlsanity.com/paper/1905.06641/) (2019)
- [Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learning](https://mlsanity.com/paper/2101.11203/) (2021)
- [On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimization](https://mlsanity.com/paper/1905.03817/) (2019)
- [Mime: Mimicking Centralized Stochastic Algorithms in Federated Learning](https://mlsanity.com/paper/2008.03606/) (2020)
- [Fast and Faster Convergence of SGD for Over-Parameterized Models and an Accelerated Perceptron](https://mlsanity.com/paper/1810.07288/) (2018)
- [SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentum](https://mlsanity.com/paper/1910.00643/) (2019)
- [Hierarchical Federated Learning Across Heterogeneous Cellular Networks](https://mlsanity.com/paper/1909.02362/) (2019)
- [Accelerating Federated Learning via Momentum Gradient Descent](https://mlsanity.com/paper/1910.03197/) (2019)
- [A Unified Analysis of Stochastic Momentum Methods for Deep Learning](https://mlsanity.com/paper/1808.10396/) (2018)
- [Hierarchical Federated Learning with Quantization: Convergence Analysis and System Design](https://mlsanity.com/paper/2103.14272/) (2021)
- [On the Convergence of Nesterov's Accelerated Gradient Method in Stochastic Settings](https://mlsanity.com/paper/2002.12414/) (2020)
- [Demystifying Why Local Aggregation Helps: Convergence Analysis of Hierarchical SGD](https://mlsanity.com/paper/2010.12998/) (2020)
- [FedADC: Accelerated Federated Learning with Drift Control](https://mlsanity.com/paper/2012.09102/) (2020)
- [Faster On-Device Training Using New Federated Momentum Algorithm](https://mlsanity.com/paper/2002.02090/) (2020)
- [Federated Learning with Nesterov Accelerated Gradient](https://mlsanity.com/paper/2009.08716/) (2020)
- [Multi-Level Local SGD for Heterogeneous Hierarchical Networks](https://mlsanity.com/paper/2007.13819/) (2020)
- [Coordinating Momenta for Cross-silo Federated Learning](https://mlsanity.com/paper/2102.03970/) (2021)

## Built on by (most cited first, 6 of 6)

- [Fast-Convergent and Communication-Alleviated Heterogeneous Hierarchical Federated Learning in Autonomous Driving](https://mlsanity.com/paper/2409.19560/) (2024)
- [FedRTS: Federated Robust Pruning via Combinatorial Thompson Sampling](https://mlsanity.com/paper/2501.19122/) (2025)
- [Recursive Offloading for LLM Serving in Multi-tier Networks](https://mlsanity.com/paper/2505.16502/) (2025)
- [Beyond Model Scale Limits: End-Edge-Cloud Federated Learning with Self-Rectified Knowledge Agglomeration](https://mlsanity.com/paper/2501.00693/) (2025)
- [A Taxonomy and Resolution Strategy for Client-Level Disagreements in Federated Learning](https://mlsanity.com/paper/2604.23386/) (2026)
- [SCOPE-FL: A Strategy-proof Chain-based Optimal pareto efficient Federated Learning System](https://mlsanity.com/paper/2606.18384/) (2026)

## Abstract (arXiv)

In this paper, we propose Hierarchical Federated Learning with Momentum Acceleration (HierMo), a three-tier worker-edge-cloud federated learning algorithm that applies momentum for training acceleration. Momentum is calculated and aggregated in the three tiers. We provide convergence analysis for HierMo, showing a convergence rate of O(1/T). In the analysis, we develop a new approach to characterize model aggregation, momentum aggregation, and their interactions. Based on this result, {we prove that HierMo achieves a tighter convergence upper bound compared with HierFAVG without momentum}. We also propose HierOPT, which optimizes the aggregation periods (worker-edge and edge-cloud aggregation periods) to minimize the loss given a limited training time.