# Introducing Noise in Decentralized Training of Neural Networks

arXiv: https://arxiv.org/abs/1809.10678 · HTML: https://mlsanity.com/paper/1809.10678/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Sep 2018.

## Built on by (most cited first, 2 of 2)

- [Robustness Enhancement in Neural Networks with Alpha-Stable Training Noise](https://mlsanity.com/paper/2311.10803/) (2023)
- [The Paradox of Noise: An Empirical Study of Noise-Infusion Mechanisms to Improve Generalization, Stability, and Privacy in Federated Learning](https://mlsanity.com/paper/2311.05790/) (2023)

## Abstract (arXiv)

It has been shown that injecting noise into the neural network weights during the training process leads to a better generalization of the resulting model. Noise injection in the distributed setup is a straightforward technique and it represents a promising approach to improve the locally trained models. We investigate the effects of noise injection into the neural networks during a decentralized training process. We show both theoretically and empirically that noise injection has no positive effect in expectation on linear models, though. However for non-linear neural networks we empirically show that noise injection substantially improves model quality helping to reach a generalization ability of a local model close to the serial baseline.