# Distributed Learning over Unreliable Networks

arXiv: https://arxiv.org/abs/1810.07766 · HTML: https://mlsanity.com/paper/1810.07766/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Oct 2018.

## Built on by (most cited first, 9 of 9)

- [A Survey of Trustworthy Federated Learning with Perspectives on Security, Robustness, and Privacy](https://mlsanity.com/paper/2302.10637/) (2023)
- [Distributed Learning Systems with First-order Methods](https://mlsanity.com/paper/2104.05245/) (2021)
- [Efficient Distributed Optimization under Heavy-Tailed Noise](https://mlsanity.com/paper/2502.04164/) (2025)
- [Personalized On-Device E-health Analytics with Decentralized Block Coordinate Descent](https://mlsanity.com/paper/2112.09341/) (2021)
- [CaBaFL: Asynchronous Federated Learning via Hierarchical Cache and Feature Balance](https://mlsanity.com/paper/2404.12850/) (2024)
- [Finite-Time Consensus Learning for Decentralized Optimization with Nonlinear Gossiping](https://mlsanity.com/paper/2111.02949/) (2021)
- [Efficient Core-selecting Incentive Mechanism for Data Sharing in Federated Learning](https://mlsanity.com/paper/2309.11722/) (2023)
- [Decentralized Learning of Tree-Structured Gaussian Graphical Models from Noisy Data](https://mlsanity.com/paper/2109.10642/) (2021)
- [ADDAI: Anomaly Detection using Distributed AI](https://mlsanity.com/paper/2205.01231/) (2022)

## Abstract (arXiv)

Most of today's distributed machine learning systems assume em reliable networks: whenever two machines exchange information (e.g., gradients or models), the network should guarantee the delivery of the message. At the same time, recent work exhibits the impressive tolerance of machine learning algorithms to errors or noise arising from relaxed communication or synchronization. In this paper, we connect these two trends, and consider the following question: em Can we design machine learning systems that are tolerant to network unreliability during training? With this motivation, we focus on a theoretical problem of independent interest---given a standard distributed parameter server architecture, if every communication between the worker and the server has a non-zero probability p of being dropped, does there exist an algorithm that still converges, and at what speed? The technical contribution of this paper is a novel theoretical analysis proving that distributed learning over unreliable network can achieve comparable convergence rate to centralized or distributed learning over reliable networks. Further, we prove that the influence of the packet drop rate diminishes with the growth of the number of textcolorblackparameter servers. We map this theoretical result onto a real-world scenario, training deep neural networks over an unreliable network layer, and conduct network simulation to validate the system improvement by allowing the networks to be unreliable.