# Delay Sensitive Hierarchical Federated Learning with Stochastic Local Updates

arXiv: https://arxiv.org/abs/2302.04851 · HTML: https://mlsanity.com/paper/2302.04851/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 24 references that are in the map.
- First posted on arXiv 9 Feb 2023.

## Builds on (most cited first, 18 of 18)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Adaptive Federated Optimization](https://mlsanity.com/paper/2003.00295/) (2020)
- [Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization](https://mlsanity.com/paper/2007.07481/) (2020)
- [Local SGD Converges Fast and Communicates Little](https://mlsanity.com/paper/1805.09767/) (2018)
- [Robust and Communication-Efficient Federated Learning from Non-IID Data](https://mlsanity.com/paper/1903.02891/) (2019)
- [FedML: A Research Library and Benchmark for Federated Machine Learning](https://mlsanity.com/paper/2007.13518/) (2020)
- [Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learning](https://mlsanity.com/paper/1807.06629/) (2018)
- [Don't Use Large Mini-Batches, Use Local SGD](https://mlsanity.com/paper/1808.07217/) (2018)
- [Client-Edge-Cloud Hierarchical Federated Learning](https://mlsanity.com/paper/1905.06641/) (2019)
- [Is Local SGD Better than Minibatch SGD?](https://mlsanity.com/paper/2002.07839/) (2020)
- [HFEL: Joint Edge Association and Resource Allocation for Cost-Efficient Hierarchical Federated Edge Learning](https://mlsanity.com/paper/2002.11343/) (2020)
- [FedVARP: Tackling the Variance Due to Partial Client Participation in Federated Learning](https://mlsanity.com/paper/2207.14130/) (2022)
- [A Unified Analysis of Federated Learning with Arbitrary Client Participation](https://mlsanity.com/paper/2205.13648/) (2022)
- [Age-Based Scheduling Policy for Federated Learning in Mobile Edge Networks](https://mlsanity.com/paper/1910.14648/) (2019)
- [Demystifying Why Local Aggregation Helps: Convergence Analysis of Hierarchical SGD](https://mlsanity.com/paper/2010.12998/) (2020)
- [Wireless Quantized Federated Learning: A Joint Computation and Communication Design](https://mlsanity.com/paper/2203.05878/) (2022)
- [Enhancing Privacy via Hierarchical Federated Learning](https://mlsanity.com/paper/2004.11361/) (2020)
- [Timely Communication in Federated Learning](https://mlsanity.com/paper/2012.15831/) (2020)

## Built on by (most cited first, 8 of 8)

- [SyncFed: Time-Aware Federated Learning through Explicit Timestamping and Synchronization](https://mlsanity.com/paper/2506.09660/) (2025)
- [Data Similarity-Based One-Shot Clustering for Multi-Task Hierarchical Federated Learning](https://mlsanity.com/paper/2410.02733/) (2024)
- [RCC-PFL: Robust Client Clustering under Noisy Labels in Personalized Federated Learning](https://mlsanity.com/paper/2503.19886/) (2025)
- [A Generalized Hierarchical Federated Learning Framework with Theoretical Guarantees](https://mlsanity.com/paper/2505.08145/) (2025)
- [Age-Aware Edge-Blind Federated Learning via Over-the-Air Aggregation](https://mlsanity.com/paper/2602.02469/) (2026)
- [A Robust Framework for Secure Cardiovascular Risk Prediction: An Architectural Case Study of Differentially Private Federated Learning](https://mlsanity.com/paper/2603.13293/) (2026)
- [Timely Asynchronous Hierarchical Federated Learning: Age of Convergence](https://mlsanity.com/paper/2306.12400/) (2023)
- [Joint Routing and Model Pruning for Decentralized Federated Learning in Bandwidth-Constrained Multi-Hop Wireless Networks](https://mlsanity.com/paper/2603.15188/) (2026)

## Abstract (arXiv)

The impact of local averaging on the performance of federated learning (FL) systems is studied in the presence of communication delay between the clients and the parameter server. To minimize the effect of delay, clients are assigned into different groups, each having its own local parameter server (LPS) that aggregates its clients' models. The groups' models are then aggregated at a global parameter server (GPS) that only communicates with the LPSs. Such setting is known as hierarchical FL (HFL). Unlike most works in the literature, the number of local and global communication rounds in our work is randomly determined by the (different) delays experienced by each group of clients. Specifically, the number of local averaging rounds is tied to a wall-clock time period coined the sync time S, after which the LPSs synchronize their models by sharing them with the GPS. Such sync time S is then reapplied until a global wall-clock time is exhausted. First, an upper bound on the deviation between the updated model at each LPS with respect to that available at the GPS is derived. This is then used as a tool to derive the convergence analysis of our proposed delay-sensitive HFL algorithm, first at each LPS individually, and then at the GPS. Our theoretical convergence bound showcases the effects of the whole system's parameters, including the number of groups, the number of clients per group, and the value of S. Our results show that the value of S should be carefully chosen, especially since it implicitly governs how the delay statistics affect the performance of HFL in situations where training time is restricted.