mlsanityData to 13 Oct 2026

Collaborative Learning over Wireless Networks: An Introductory Overview

arXiv 2112.05559 ↗First posted 7 Dec 2021cs.LG cs.DC eess.SPcited by 4

Not on the map. Why?

Abstract (arXiv)

In this chapter, we will mainly focus on collaborative training across wireless devices. Training a ML model is equivalent to solving an optimization problem, and many distributed optimization algorithms have been developed over the last decades. These distributed ML algorithms provide data locality; that is, a joint model can be trained collaboratively while the data available at each participating device remains local. This addresses, to some extend, the privacy concern. They also provide computational scalability as they allow exploiting computational resources distributed across many edge devices. However, in practice, this does not directly lead to a linear gain in the overall learning speed with the number of devices. This is partly due to the communication bottleneck limiting the overall computation speed. Additionally, wireless devices are highly heterogeneous in their computational capabilities, and both their computation speed and communication rate can be highly time-varying due to physical factors. Therefore, distributed learning algorithms, particularly those to be implemented at the wireless network edge, must be carefully designed taking into account the impact of time-varying communication network as well as the heterogeneous and stochastic computation capabilities of devices.

Key facts. Cited by 4 papers in the map (data through 13 Oct 2026). Builds on 56 of its 64 references that are in the map. First posted on arXiv 7 Dec 2021.

Builds on

56 of 64 references.

References · 56

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,950
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,321
  3. 2020Adaptive Federated Optimizationcited by 1,041
  4. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  5. 2020The Future of Digital Health with Federated Learningcited by 630
  6. 2018Local SGD Converges Fast and Communicates Littlecited by 601
  7. 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 567
  8. 2020Federated Learning with Matched Averagingcited by 564
  9. 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 500
  10. 2017Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Trainingcited by 478
  11. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 419
  12. 2018Sparsified SGD with Memorycited by 376
38 more
  1. 2018Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edgecited by 350
  2. 2019Error Feedback Fixes SignSGD and other Gradient Compression Schemescited by 308
  3. 2018Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learningcited by 278
  4. 2017TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learningcited by 256
  5. 2017Sparse Communication for Distributed Gradient Descentcited by 249
  6. 2018Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Datacited by 243
  7. 2018The Convergence of Sparsified Gradient Methodscited by 229
  8. 2018Don't Use Large Mini-Batches, Use Local SGDcited by 202
  9. 2019Client-Edge-Cloud Hierarchical Federated Learningcited by 202
  10. 2017Gradient Sparsification for Communication-Efficient Distributed Optimizationcited by 201
  11. 2019Federated Learning over Wireless Fading Channelscited by 182
  12. 2019Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computationscited by 181
  13. 2019On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimizationcited by 179
  14. 2019Privacy-preserving Federated Brain Tumour Segmentationcited by 169
  15. 2013On the Convergence of Decentralized Gradient Descentcited by 158
  16. 2016QSGD: Communication-Efficient SGD via Gradient Quantization and Encodingcited by 147
  17. 2018D2: Decentralized Training over Decentralized Datacited by 140
  18. 2019Scheduling Policies for Federated Learning in Wireless Networkscited by 131
  19. 2019SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentumcited by 121
  20. 2015Asynchronous Parallel Stochastic Gradient for Nonconvex Optimizationcited by 115
  21. 2019Local SGD with Periodic Averaging: Tighter Analysis and Adaptive Synchronizationcited by 107
  22. 2018Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithmscited by 106
  23. 2019DoubleSqueeze: Parallel Stochastic Gradient Descent with Double-Pass Error-Compensated Compressioncited by 99
  24. 2016Asynchronous Stochastic Gradient Descent with Delay Compensationcited by 98
  25. 2019Hierarchical Federated Learning Across Heterogeneous Cellular Networkscited by 98
  26. 2017On the convergence properties of a K-step averaging stochastic gradient descent algorithm for nonconvex optimizationcited by 92
  27. 2018Wireless Network Intelligence at the Edgecited by 92
  28. 2018Error Compensated Quantized SGD and its Applications to Large-scale Distributed Optimizationcited by 76
  29. 2020Convergence of Update Aware Device Scheduling for Federated Learning at the Wireless Edgecited by 71
  30. 2017Collaborative Deep Learning in Fixed Topology Networkscited by 60
  31. 2020Multi-Armed Bandit Based Client Scheduling for Federated Learningcited by 56
  32. 2019Communication-Efficient Distributed Blockwise Momentum SGD with Error-Feedbackcited by 50
  33. 2019Device Scheduling with Fast Convergence for Wireless Federated Learningcited by 46
  34. 2015Staleness-aware Async-SGD for Distributed Deep Learningcited by 43
  35. 2016On Nonconvex Decentralized Gradient Descentcited by 42
  36. 2019Age-Based Scheduling Policy for Federated Learning in Mobile Edge Networkscited by 39
  37. 2020Energy-Efficient Resource Management for Federated Edge Learning with CPU-GPU Heterogeneous Computingcited by 39
  38. 2019A Distributed Synchronous SGD Algorithm with Global Top-k Sparsification for Low Bandwidth Networkscited by 37

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 0; 2024: 0; 2025: 2.

4 papers.

  1. 2022Semi-Decentralized Federated Learning with Collaborative Relayingcited by 16
  2. 2022Robust Federated Learning with Connectivity Failures: A Semi-Decentralized Framework with Collaborative Relayingcited by 5
  3. 2025MAB-Based Channel Scheduling for Asynchronous Federated Learning in Non-Stationary Environmentscited by 0
  4. 2025Controlled privacy leakage propagation throughout overlapping grouped learningcited by 0