# Robustness Enhancement in Neural Networks with Alpha-Stable Training Noise

arXiv: https://arxiv.org/abs/2311.10803 · HTML: https://mlsanity.com/paper/2311.10803/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 58 references that are in the map.
- First posted on arXiv 17 Nov 2023.

## Builds on (most cited first, 15 of 15)

- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Universal adversarial perturbations](https://mlsanity.com/paper/1610.08401/) (2016)
- [Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks](https://mlsanity.com/paper/2102.00554/) (2021)
- [Variational Dropout Sparsifies Deep Neural Networks](https://mlsanity.com/paper/1701.05369/) (2017)
- [MNIST-C: A Robustness Benchmark for Computer Vision](https://mlsanity.com/paper/1906.02337/) (2019)
- [Robustness of classifiers: from adversarial to random noise](https://mlsanity.com/paper/1608.08967/) (2016)
- [Analysis of classifiers' robustness to adversarial perturbations](https://mlsanity.com/paper/1502.02590/) (2015)
- [Efficient Defenses Against Adversarial Attacks](https://mlsanity.com/paper/1707.06728/) (2017)
- [On the Heavy-Tailed Theory of Stochastic Gradient Descent for Deep Neural Networks](https://mlsanity.com/paper/1912.00018/) (2019)
- [A simple way to make neural networks robust against diverse image corruptions](https://mlsanity.com/paper/2001.06057/) (2020)
- [Improving Model Robustness by Adaptively Correcting Perturbation Levels with Active Queries](https://mlsanity.com/paper/2103.14824/) (2021)
- [Robustness of convolutional neural networks to physiological ECG noise](https://mlsanity.com/paper/2108.01995/) (2021)
- [Black-Box Decision based Adversarial Attack with Symmetric α-stable Distribution](https://mlsanity.com/paper/1904.05586/) (2019)
- [Introducing Noise in Decentralized Training of Neural Networks](https://mlsanity.com/paper/1809.10678/) (2018)

## Built on by (most cited first, 2 of 2)

- [Uncertainty Quantification With Noise Injection in Neural Networks: A Bayesian Perspective](https://mlsanity.com/paper/2501.12314/) (2025)
- [Improving Out-of-Distribution Data Handling and Corruption Resistance via Modern Hopfield Networks](https://mlsanity.com/paper/2408.11309/) (2024)

## Abstract (arXiv)

With the increasing use of deep learning on data collected by non-perfect sensors and in non-perfect environments, the robustness of deep learning systems has become an important issue. A common approach for obtaining robustness to noise has been to train deep learning systems with data augmented with Gaussian noise. In this work, we challenge the common choice of Gaussian noise and explore the possibility of stronger robustness for non-Gaussian impulsive noise, specifically alpha-stable noise. Justified by the Generalized Central Limit Theorem and evidenced by observations in various application areas, alpha-stable noise is widely present in nature. By comparing the testing accuracy of models trained with Gaussian noise and alpha-stable noise on data corrupted by different noise, we find that training with alpha-stable noise is more effective than Gaussian noise, especially when the dataset is corrupted by impulsive noise, thus improving the robustness of the model. The generality of this conclusion is validated through experiments conducted on various deep learning models with image and time series datasets, and other benchmark corrupted datasets. Consequently, we propose a novel data augmentation method that replaces Gaussian noise, which is typically added to the training data, with alpha-stable noise.