# Training neural networks with structured noise improves classification and generalization

arXiv: https://arxiv.org/abs/2302.13417 · HTML: https://mlsanity.com/paper/2302.13417/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 74 references that are in the map.
- First posted on arXiv 26 Feb 2023.

## Builds on (most cited first, 11 of 11)

- [CVXPY: A Python-Embedded Modeling Language for Convex Optimization](https://mlsanity.com/paper/1603.00943/) (2016)
- [The Forward-Forward Algorithm: Some Preliminary Investigations](https://mlsanity.com/paper/2212.13345/) (2022)
- [Information Dropout: Learning Optimal Representations Through Noisy Computation](https://mlsanity.com/paper/1611.01353/) (2016)
- [Maximum-Entropy Adversarial Data Augmentation for Improved Generalization and Robustness](https://mlsanity.com/paper/2010.08001/) (2020)
- [Regularizing Deep Neural Networks by Noise: Its Interpretation and Optimization](https://mlsanity.com/paper/1710.05179/) (2017)
- [Survey of Dropout Methods for Deep Neural Networks](https://mlsanity.com/paper/1904.13310/) (2019)
- [The Overfitted Brain: Dreams evolved to assist generalization](https://mlsanity.com/paper/2007.09560/) (2020)
- [Learning cortical representations through perturbed and adversarial dreaming](https://mlsanity.com/paper/2109.04261/) (2021)
- [Supervised perceptron learning vs unsupervised Hebbian unlearning: Approaching optimal memory retrieval in Hopfield-like networks](https://mlsanity.com/paper/2201.00032/) (2021)
- [Categorical Perception: A Groundwork for Deep Learning](https://mlsanity.com/paper/2012.05549/) (2020)
- [How deep convolutional neural networks lose spatial information with training](https://mlsanity.com/paper/2210.01506/) (2022)

## Built on by (most cited first, 4 of 4)

- [Deterministic versus stochastic dynamical classifiers: opposing random adversarial attacks with noise](https://mlsanity.com/paper/2409.13470/) (2024)
- [RIVAL: Reinforcement Learning with Iterative and Adversarial Optimization for Machine Translation](https://mlsanity.com/paper/2506.05070/) (2025)
- [An Empirical Study on Fault Detection and Root Cause Analysis of Indium Tin Oxide Electrodes by Processing S-parameter Patterns](https://mlsanity.com/paper/2308.11639/) (2023)
- [Demolition and Reinforcement of Memories in Spin-Glass-like Neural Networks](https://mlsanity.com/paper/2403.02537/) (2024)

## Abstract (arXiv)

The beneficial role of noise-injection in learning is a consolidated concept in the field of artificial neural networks, suggesting that even biological systems might take advantage of similar mechanisms to optimize their performance. The training-with-noise algorithm proposed by Gardner and collaborators is an emblematic example of a noise-injection procedure in recurrent networks, which can be used to model biological neural systems. We show how adding structure to noisy training data can substantially improve the algorithm performance, allowing the network to approach perfect retrieval of the memories and wide basins of attraction, even in the scenario of maximal injected noise. We also prove that the so-called Hebbian Unlearning rule coincides with the training-with-noise algorithm when noise is maximal and data are stable fixed points of the network dynamics.