# Distributed Machine Learning through Heterogeneous Edge Systems

arXiv: https://arxiv.org/abs/1911.06949 · HTML: https://mlsanity.com/paper/1911.06949/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Nov 2019.

## Built on by (most cited first, 3 of 3)

- [dPRO: A Generic Profiling and Optimization System for Expediting Distributed DNN Training](https://mlsanity.com/paper/2205.02473/) (2022)
- [Bristle: Decentralized Federated Learning in Byzantine, Non-i.i.d. Environments](https://mlsanity.com/paper/2110.11006/) (2021)
- [Edge Deep Learning in Computer Vision and Medical Diagnostics: A Comprehensive Survey](https://mlsanity.com/paper/2605.06714/) (2026)

## Abstract (arXiv)

Many emerging AI applications request distributed machine learning (ML) among edge systems (e.g., IoT devices and PCs at the edge of the Internet), where data cannot be uploaded to a central venue for model training, due to their large volumes and/or security/privacy concerns. Edge devices are intrinsically heterogeneous in computing capacity, posing significant challenges to parameter synchronization for parallel training with the parameter server (PS) architecture. This paper proposes ADSP, a parameter synchronization scheme for distributed machine learning (ML) with heterogeneous edge systems. Eliminating the significant waiting time occurring with existing parameter synchronization models, the core idea of ADSP is to let faster edge devices continue training, while committing their model updates at strategically decided intervals. We design algorithms that decide time points for each worker to commit its model update, and ensure not only global model convergence but also faster convergence. Our testbed implementation and experiments show that ADSP outperforms existing parameter synchronization models significantly in terms of ML model convergence time, scalability and adaptability to large heterogeneity.