# Impact of white Gaussian internal noise on analog echo-state neural networks

arXiv: https://arxiv.org/abs/2405.07670 · HTML: https://mlsanity.com/paper/2405.07670/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 44 references that are in the map.
- First posted on arXiv 13 May 2024.

## Builds on (most cited first, 9 of 9)

- [All-Optical Machine Learning Using Diffractive Deep Neural Networks](https://mlsanity.com/paper/1804.08711/) (2018)
- [Recent Advances in Recurrent Neural Networks](https://mlsanity.com/paper/1801.01078/) (2017)
- [Transparency of Deep Neural Networks for Medical Image Analysis: A Review of Interpretability Methods](https://mlsanity.com/paper/2111.02398/) (2021)
- [Reinforcement Learning in a large scale photonic Recurrent Neural Network](https://mlsanity.com/paper/1711.05133/) (2017)
- [An optical neural network using less than 1 photon per multiplication](https://mlsanity.com/paper/2104.13467/) (2021)
- [Understanding and mitigating noise in trained deep neural networks](https://mlsanity.com/paper/2103.07413/) (2021)
- [Quantum-limited stochastic optical neural networks operating at a few quanta per activation](https://mlsanity.com/paper/2307.15712/) (2023)
- [Data-driven Modeling of Mach-Zehnder Interferometer-based Optical Matrix Multipliers](https://mlsanity.com/paper/2210.09171/) (2022)
- [Noise impact on recurrent neural network with linear activation function](https://mlsanity.com/paper/2303.13262/) (2023)

## Abstract (arXiv)

In recent years, more and more works have appeared devoted to the analog (hardware) implementation of artificial neural networks, in which neurons and the connection between them are based not on computer calculations, but on physical principles. Such networks offer improved energy efficiency and, in some cases, scalability, but may be susceptible to internal noise. This paper studies the influence of noise on the functioning of recurrent networks using the example of trained echo state networks (ESNs). The most common reservoir connection matrices were chosen as various topologies of ESNs: random uniform and band matrices with different connectivity. White Gaussian noise was chosen as the influence, and according to the way of its introducing it was additive or multiplicative, as well as correlated or uncorrelated. In the paper, we show that the propagation of noise in reservoir is mainly controlled by the statistical properties of the output connection matrix, namely the mean and the mean square. Depending on these values, more correlated or uncorrelated noise accumulates in the network. We also show that there are conditions under which even noise with an intensity of 10-20 is already enough to completely lose the useful signal. In the article we show which types of noise are most critical for networks with different activation functions (hyperbolic tangent, sigmoid and linear) and if the network is self-closed.