# Internal noise in hardware deep and recurrent neural networks helps with learning

arXiv: https://arxiv.org/abs/2504.13778 · HTML: https://mlsanity.com/paper/2504.13778/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Apr 2025.

## Built on by (most cited first, 1 of 1)

- [Intrinsic-Noise Consolidation: A Doob-Barrier-Conditioned Diffusion Turns Analog Device Noise into a Continual-Learning Resource](https://mlsanity.com/paper/2607.06924/) (2026)

## Abstract (arXiv)

Recently, the field of hardware neural networks has been actively developing, where neurons and their connections are not simulated on a computer but are implemented at the physical level, transforming the neural network into a tangible device. In this paper, we investigate how internal noise during the training of neural networks affects the final performance of recurrent and deep neural networks. We consider feedforward networks (FNN) and echo state networks (ESN) as examples. The types of noise examined originated from a real optical implementation of a neural network. However, these types were subsequently generalized to enhance the applicability of our findings on a broader scale. The noise types considered include additive and multiplicative noise, which depend on how noise influences each individual neuron, and correlated and uncorrelated noise, which pertains to the impact of noise on groups of neurons (such as the hidden layer of FNNs or the reservoir of ESNs). In this paper, we demonstrate that, in most cases, both deep and echo state networks benefit from internal noise during training, as it enhances their resilience to noise. Consequently, the testing performance at the same noise intensities is significantly higher for networks trained with noise than for those trained without it. Notably, only multiplicative correlated noise during training has minimal has almost no impact on both deep and recurrent networks.