# Network Support for High-performance Distributed Machine Learning

arXiv: https://arxiv.org/abs/2102.03394 · HTML: https://mlsanity.com/paper/2102.03394/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 52 references that are in the map.
- First posted on arXiv 5 Feb 2021.

## Builds on (most cited first, 19 of 19)

- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [Revisiting Unreasonable Effectiveness of Data in Deep Learning Era](https://mlsanity.com/paper/1707.02968/) (2017)
- [Deep Learning Scaling is Predictable, Empirically](https://mlsanity.com/paper/1712.00409/) (2017)
- [Federated Learning for Mobile Keyboard Prediction](https://mlsanity.com/paper/1811.03604/) (2018)
- [Adaptive Federated Learning in Resource Constrained Edge Computing Systems](https://mlsanity.com/paper/1804.05271/) (2018)
- [Federated Optimization:Distributed Optimization Beyond the Datacenter](https://mlsanity.com/paper/1511.03575/) (2015)
- [Network Topology and Communication-Computation Tradeoffs in Decentralized Optimization](https://mlsanity.com/paper/1709.08765/) (2017)
- [Submodular Optimization with Submodular Cover and Submodular Knapsack Constraints](https://mlsanity.com/paper/1311.2106/) (2013)
- [Federated Deep Reinforcement Learning](https://mlsanity.com/paper/1901.08277/) (2019)
- [Cooperative and Distributed Reinforcement Learning of Drones for Field Coverage](https://mlsanity.com/paper/1803.07250/) (2018)
- [Active Learning for Wireless IoT Intrusion Detection](https://mlsanity.com/paper/1808.01412/) (2018)
- [Towards Self-learning Edge Intelligence in 6G](https://mlsanity.com/paper/2010.00176/) (2020)
- [Near-Optimal Straggler Mitigation for Distributed Gradient Methods](https://mlsanity.com/paper/1710.09990/) (2017)
- [Communication-Efficient Policy Gradient Methods for Distributed Reinforcement Learning](https://mlsanity.com/paper/1812.03239/) (2018)
- [Distributed Deep Q-Learning](https://mlsanity.com/paper/1508.04186/) (2015)
- [Impact of Training Dataset Size on Neural Answer Selection Models](https://mlsanity.com/paper/1901.10496/) (2019)
- [Active Learning-based Classification in Automated Connected Vehicles](https://mlsanity.com/paper/2002.07593/) (2020)
- [ASAP: Asynchronous Approximate Data-Parallel Computation](https://mlsanity.com/paper/1612.08608/) (2016)

## Built on by (most cited first, 4 of 4)

- [Explainable AI over the Internet of Things (IoT): Overview, State-of-the-Art and Future Directions](https://mlsanity.com/paper/2211.01036/) (2022)
- [Choose, not Hoard: Information-to-Model Matching for Artificial Intelligence in O-RAN](https://mlsanity.com/paper/2208.04229/) (2022)
- [Unexpectedly Useful: Convergence Bounds And Real-World Distributed Learning](https://mlsanity.com/paper/2212.02155/) (2022)
- [Priority-Aware Learning-Unlearning Correction for Dynamic Decentralized LoRA Fine-Tuning](https://mlsanity.com/paper/2606.22878/) (2026)

## Abstract (arXiv)

The traditional approach to distributed machine learning is to adapt learning algorithms to the network, e.g., reducing updates to curb overhead. Networks based on intelligent edge, instead, make it possible to follow the opposite approach, i.e., to define the logical network topology em around the learning task to perform, so as to meet the desired learning performance. In this paper, we propose a system model that captures such aspects in the context of supervised machine learning, accounting for both learning nodes (that perform computations) and information nodes (that provide data). We then formulate the problem of selecting (i) which learning and information nodes should cooperate to complete the learning task, and (ii) the number of iterations to perform, in order to minimize the learning cost while meeting the target prediction error and execution time. After proving important properties of the above problem, we devise an algorithm, named DoubleClimb, that can find a 1+1/|I|-competitive solution (with I being the set of information nodes), with cubic worst-case complexity. Our performance evaluation, leveraging a real-world network topology and considering both classification and regression tasks, also shows that DoubleClimb closely matches the optimum, outperforming state-of-the-art alternatives.