# Design and Optimization of Hierarchical Gradient Coding for Distributed Learning at Edge Devices

arXiv: https://arxiv.org/abs/2406.10831 · HTML: https://mlsanity.com/paper/2406.10831/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 40 references that are in the map.
- First posted on arXiv 16 Jun 2024.

## Builds on (most cited first, 21 of 21)

- [Edge Intelligence: Paving the Last Mile of Artificial Intelligence with Edge Computing](https://mlsanity.com/paper/1905.10083/) (2019)
- [Speeding Up Distributed Machine Learning Using Codes](https://mlsanity.com/paper/1512.02673/) (2015)
- [HFEL: Joint Edge Association and Resource Allocation for Cost-Efficient Hierarchical Federated Edge Learning](https://mlsanity.com/paper/2002.11343/) (2020)
- [Energy-Efficient Radio Resource Allocation for Federated Edge Learning](https://mlsanity.com/paper/1907.06040/) (2019)
- [Coded Computing for Low-Latency Federated Learning over Wireless Edge Networks](https://mlsanity.com/paper/2011.06223/) (2020)
- [Energy-Efficient Federated Edge Learning with Joint Communication and Computation Design](https://mlsanity.com/paper/2003.00199/) (2020)
- [Communication-Computation Efficient Gradient Coding](https://mlsanity.com/paper/1802.03475/) (2018)
- [CodedPaddedFL and CodedSecAgg: Straggler Mitigation and Secure Aggregation in Federated Learning](https://mlsanity.com/paper/2112.08909/) (2021)
- [Gradient Coding from Cyclic MDS Codes and Expander Graphs](https://mlsanity.com/paper/1707.03858/) (2017)
- [DReS-FL: Dropout-Resilient Secure Federated Learning for Non-IID Clients via Secret Data Sharing](https://mlsanity.com/paper/2210.02680/) (2022)
- [Joint Parameter-and-Bandwidth Allocation for Improving the Efficiency of Partitioned Edge Learning](https://mlsanity.com/paper/2003.04544/) (2020)
- [Improving Distributed Gradient Descent Using Reed-Solomon Codes](https://mlsanity.com/paper/1706.05436/) (2017)
- [Gradient Coding with Clustering and Multi-message Communication](https://mlsanity.com/paper/1903.01974/) (2019)
- [HierTrain: Fast Hierarchical Edge AI Learning with Hybrid Parallelism in Mobile-Edge-Cloud Computing](https://mlsanity.com/paper/2003.09876/) (2020)
- [Gradient Coding with Dynamic Clustering for Straggler-Tolerant Distributed Learning](https://mlsanity.com/paper/2103.01206/) (2021)
- [In-network Computation for Large-scale Federated Learning over Wireless Edge Networks](https://mlsanity.com/paper/2109.10903/) (2021)
- [Heterogeneity-aware Gradient Coding for Straggler Tolerance](https://mlsanity.com/paper/1901.09339/) (2019)
- [Joint Coding and Scheduling Optimization for Distributed Learning over Wireless Edge Networks](https://mlsanity.com/paper/2103.04303/) (2021)
- [Coded Fourier Transform](https://mlsanity.com/paper/1710.06471/) (2017)
- [Coded Gradient Aggregation: A Tradeoff Between Communication Costs at Edge Nodes and at Helper Nodes](https://mlsanity.com/paper/2105.02919/) (2021)
- [Jointly Optimize Coding and Node Selection for Distributed Computing over Wireless Edge Networks](https://mlsanity.com/paper/2106.05475/) (2021)

## Abstract (arXiv)

Edge computing has recently emerged as a promising paradigm to boost the performance of distributed learning by leveraging the distributed resources at edge nodes. Architecturally, the introduction of edge nodes adds an additional intermediate layer between the master and workers in the original distributed learning systems, potentially leading to more severe straggler effect. Recently, coding theory-based approaches have been proposed for stragglers mitigation in distributed learning, but the majority focus on the conventional workers-master architecture. In this paper, along a different line, we investigate the problem of mitigating the straggler effect in hierarchical distributed learning systems with an additional layer composed of edge nodes. Technically, we first derive the fundamental trade-off between the computational loads of workers and the stragglers tolerance. Then, we propose a hierarchical gradient coding framework, which provides better stragglers mitigation, to achieve the derived computational trade-off. To further improve the performance of our framework in heterogeneous scenarios, we formulate an optimization problem with the objective of minimizing the expected execution time for each iteration in the learning process. We develop an efficient algorithm to mathematically solve the problem by outputting the optimum strategy. Extensive simulation results demonstrate the superiority of our schemes compared with conventional solutions.