mlsanityData to 7 Oct 2026

Gossip training for deep learning

arXiv 1611.09726 ↗First posted 29 Nov 2016cs.CV cs.LG stat.MLcited by 33

Not on the map. Why?

Abstract (arXiv)

We address the issue of speeding up the training of convolutional networks. Here we study a distributed method adapted to stochastic gradient descent (SGD). The parallel optimization setup uses several threads, each applying individual gradient descents on a local variable. We propose a new way to share information between different threads inspired by gossip algorithms and showing good consensus convergence properties. Our method called GoSGD has the advantage to be fully asynchronous and decentralized. We compared our method to the recent EASGD in citeelastic on CIFAR-10 show encouraging results.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Nov 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 8; 2023: 7; 2024: 9; 2025: 3; 2026: 2.

33 papers.

Sort:

  1. 2022DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Trainingcited by 74
  2. 2021Decentralized federated learning of deep neural networks on non-iid datacited by 23
  3. 2022An Energy and Carbon Footprint Analysis of Distributed and Federated Learningcited by 16
  4. 2024Decentralized Directed Collaboration for Personalized Federated Learningcited by 9
  5. 2021A Framework for Energy and Carbon Footprint Analysis of Distributed and Federated Edge Learningcited by 8
  6. 2023Towards More Suitable Personalization in Federated Learning via Decentralized Partial Model Trainingcited by 8
  7. 2022Decentralized adaptive clustering of deep nets is beneficial for client collaborationcited by 7
  8. 2023A2CiD2: Accelerating Asynchronous Communication in Decentralized Deep Learningcited by 7
  9. 2022Network Gradient Descent Algorithm for Decentralized Federated Learningcited by 5
  10. 2022Olive Branch Learning: A Topology-Aware Federated Learning Framework for Space-Air-Ground Integrated Networkcited by 5
  11. 2024DRACO: Decentralized Asynchronous Federated Learning over Row-Stochastic Wireless Networkscited by 5
  12. 2025OmniLearn: A Framework for Distributed Deep Learning over Heterogeneous Clusterscited by 5
21 more
  1. 2023Efficient Node Selection in Private Personalized Decentralized Learningcited by 4
  2. 2025NoLoCo: No-all-reduce Low Communication Training Method for Large Modelscited by 4
  3. 2026NuMuon: Nuclear-Norm-Constrained Muon for Compressible LLM Trainingcited by 4
  4. 2023Tram-FL: Routing-based Model Training for Decentralized Federated Learningcited by 3
  5. 2024Protocol Learning, Decentralized Frontier Risk and the No-Off Problemcited by 3
  6. 2021DeceFL: A Principled Decentralized Federated Learning Frameworkcited by 2
  7. 2022Asynchronous Training Schemes in Distributed Learning with Time Delaycited by 2
  8. 2022Accelerating Parallel Stochastic Gradient Descent via Non-blocking Mini-batchescited by 2
  9. 2021EventGraD: Event-Triggered Communication in Parallel Machine Learningcited by 1
  10. 2022RCD-SGD: Resource-Constrained Distributed SGD in Heterogeneous Environment via Submodular Partitioningcited by 1
  11. 2023Structured Cooperative Learning with Graphical Model Priorscited by 1
  12. 2023Collaborative Distributed Machine Learningcited by 1
  13. 2024Adjacent Leader Decentralized Stochastic Gradient Descentcited by 1
  14. 2024On the effects of similarity metrics in decentralized deep learning under distributional shiftcited by 1
  15. 2023Concept-aware clustering for decentralized deep learning under temporal shiftcited by 0
  16. 2024Technical Report: On the Convergence of Gossip Learning in the Presence of Node Inaccessibilitycited by 0
  17. 2024A Selective Review on Statistical Methods for Massive Data Computation: Distributed Computing, Subsampling, and Minibatch Techniquescited by 0
  18. 2024Optimally Improving Cooperative Learning in a Social Settingcited by 0
  19. 2024Sequence-to-sequence models in peer-to-peer learning: A practical applicationcited by 0
  20. 2025Load-Aware Training Scheduling for Model Circulation-based Decentralized Federated Learningcited by 0
  21. 2026Unextractable Protocol Models: Collaborative Training and Inference without Weight Materializationcited by 0