# Helios: Heterogeneity-Aware Federated Learning with Dynamically Balanced Collaboration

arXiv: https://arxiv.org/abs/1912.01684 · HTML: https://mlsanity.com/paper/1912.01684/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Dec 2019.

## Built on by (most cited first, 15 of 15)

- [Federated Learning for Computationally-Constrained Heterogeneous Devices: A Survey](https://mlsanity.com/paper/2307.09182/) (2023)
- [Every Parameter Matters: Ensuring the Convergence of Federated Learning with Dynamic Heterogeneous Models Reduction](https://mlsanity.com/paper/2310.08670/) (2023)
- [Architectural Patterns for the Design of Federated Learning Systems](https://mlsanity.com/paper/2101.02373/) (2021)
- [Asteroid: Resource-Efficient Hybrid Pipeline Parallelism for Collaborative DNN Training on Heterogeneous Edge Devices](https://mlsanity.com/paper/2408.08015/) (2024)
- [DISTREAL: Distributed Resource-Aware Learning in Heterogeneous Systems](https://mlsanity.com/paper/2112.08761/) (2021)
- [Critical Learning Periods in Federated Learning](https://mlsanity.com/paper/2109.05613/) (2021)
- [Stochastic Coded Federated Learning: Theoretical Analysis and Incentive Mechanism Design](https://mlsanity.com/paper/2211.04132/) (2022)
- [Efficient Federated Learning with Enhanced Privacy via Lottery Ticket Pruning in Edge Computing](https://mlsanity.com/paper/2305.01387/) (2023)
- [Federated Action Recognition on Heterogeneous Embedded Devices](https://mlsanity.com/paper/2107.12147/) (2021)
- [On the Convergence of Heterogeneous Federated Learning with Arbitrary Adaptive Online Model Pruning](https://mlsanity.com/paper/2201.11803/) (2022)
- [FedSS: Federated Learning with Smart Selection of clients](https://mlsanity.com/paper/2207.04569/) (2022)
- [Latency Aware Semi-synchronous Client Selection and Model Aggregation for Wireless Federated Learning](https://mlsanity.com/paper/2210.10311/) (2022)
- [Nothing Wasted: Full Contribution Enforcement in Federated Edge Learning](https://mlsanity.com/paper/2110.08330/) (2021)
- [Revolutionizing Wireless Networks with Federated Learning: A Comprehensive Review](https://mlsanity.com/paper/2308.04404/) (2023)
- [Toward efficient resource utilization at edge nodes in federated learning](https://mlsanity.com/paper/2309.10367/) (2023)

## Abstract (arXiv)

In this paper, we propose Helios, a heterogeneity-aware FL framework to tackle the straggler issue. Helios identifies individual devices' heterogeneous training capability, and therefore the expected neural network model training volumes regarding the collaborative training pace. For straggling devices, a "soft-training" method is proposed to dynamically compress the original identical training model into the expected volume through a rotating neuron training approach. With extensive algorithm analysis and optimization schemes, the stragglers can be accelerated while retaining the convergence for local training as well as federated collaboration.