# GoSGD: Distributed Optimization for Deep Learning with Gossip Exchange

arXiv: https://arxiv.org/abs/1804.01852 · HTML: https://mlsanity.com/paper/1804.01852/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Apr 2018.

## Built on by (most cited first, 6 of 6)

- [Implicit Model Specialization through DAG-based Decentralized Federated Learning](https://mlsanity.com/paper/2111.01257/) (2021)
- [Enabling All In-Edge Deep Learning: A Literature Review](https://mlsanity.com/paper/2204.03326/) (2022)
- [Detection of Insider Attacks in Distributed Projected Subgradient Algorithms](https://mlsanity.com/paper/2101.06917/) (2021)
- [Consensus Based Multi-Layer Perceptrons for Edge Computing](https://mlsanity.com/paper/2102.05021/) (2021)
- [Non asymptotic bounds in asynchronous sum-weight gossip protocols](https://mlsanity.com/paper/2111.10248/) (2021)
- [On the Limit Performance of Floating Gossip](https://mlsanity.com/paper/2302.08413/) (2023)

## Abstract (arXiv)

We address the issue of speeding up the training of convolutional neural networks by studying a distributed method adapted to stochastic gradient descent. Our parallel optimization setup uses several threads, each applying individual gradient descents on a local variable. We propose a new way of sharing information between different threads based on gossip algorithms that show good consensus convergence properties. Our method called GoSGD has the advantage to be fully asynchronous and decentralized.