mlsanityData to 7 Oct 2026

Noise Optimization for Artificial Neural Networks

arXiv 2102.04450 ↗First posted 6 Feb 2021cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Adding noises to artificial neural network(ANN) has been shown to be able to improve robustness in previous work. In this work, we propose a new technique to compute the pathwise stochastic gradient estimate with respect to the standard deviation of the Gaussian noise added to each neuron of the ANN. By our proposed technique, the gradient estimate with respect to noise levels is a byproduct of the backpropagation algorithm for estimating gradient with respect to synaptic weights in ANN. Thus, the noise level for each neuron can be optimized simultaneously in the processing of training the synaptic weights at nearly no extra computational cost. In numerical experiments, our proposed method can achieve significant performance improvement on robustness of several popular ANN structures under both black box and white box attacks tested in various computer vision datasets.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 35 of its 54 references that are in the map. First posted on arXiv 6 Feb 2021.

Builds on

35 of 54 references.

References · 35

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2013Intriguing properties of neural networkscited by 5,114
  4. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  5. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  6. 2017SmoothGrad: removing noise by adding noisecited by 1,097
  7. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 897
  8. 2016Practical Black-Box Attacks against Machine Learningcited by 893
  9. 2017Ensemble Adversarial Training: Attacks and Defensescited by 833
  10. 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 675
  11. 2020On Adaptive Attacks to Adversarial Example Defensescited by 547
  12. 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 542
23 more
  1. 2017Defense against Adversarial Attacks Using High-Level Representation Guided Denoisercited by 329
  2. 2017Parseval Networks: Improving Robustness to Adversarial Examplescited by 288
  3. 2019Monte Carlo Gradient Estimation in Machine Learningcited by 281
  4. 2019Simple Black-box Adversarial Attackscited by 249
  5. 2017Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradientscited by 236
  6. 2018Why do deep convolutional networks generalize so poorly to small image transformations?cited by 198
  7. 2016A study of the effect of JPG compression on adversarial imagescited by 196
  8. 2018Stochastic Activation Pruning for Robust Adversarial Defensecited by 171
  9. 2017Towards Robust Neural Networks via Random Self-ensemblecited by 158
  10. 2016Improving the Robustness of Deep Neural Networks via Stability Trainingcited by 156
  11. 2017The Space of Transferable Adversarial Examplescited by 156
  12. 2015Adding Gradient Noise Improves Learning for Very Deep Networkscited by 144
  13. 2016Examining the Impact of Blur on Recognition by Convolutional Networkscited by 58
  14. 2016Noisy Activation Functionscited by 53
  15. 2019What it Thinks is Important is Important: Robustness Transfers through Input Gradientscited by 34
  16. 2019Cubic LSTMs for Video Predictioncited by 32
  17. 2018Adversarial Noise Layer: Regularize Neural Network By Adding Noisecited by 25
  18. 2018Estimate and Replace: A Novel Approach to Integrating Deep Neural Networks with Existing Applicationscited by 21
  19. 2019Benchmarking Adversarial Robustnesscited by 15
  20. 2019A Kernelized Manifold Mapping to Diminish the Effect of Adversarial Perturbationscited by 11
  21. 2019Measuring the Transferability of Adversarial Examplescited by 6
  22. 2019White Noise Analysis of Neural Networkscited by 4
  23. 2019Potential adversarial samples for white-box attackscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 3; 2024: 1; 2025: 1; 2026: 1.

7 papers.

  1. 2023One Forward is Enough for Neural Network Training via Likelihood Ratio Methodcited by 11
  2. 2023A Novel Noise Injection-based Training Scheme for Better Model Robustnesscited by 3
  3. 2022Noise Injection Node Regularization for Robust Learningcited by 1
  4. 2023The Paradox of Noise: An Empirical Study of Noise-Infusion Mechanisms to Improve Generalization, Stability, and Privacy in Federated Learningcited by 1
  5. 2025Statistical Quality and Reproducibility of Pseudorandom Number Generators in Machine Learning technologiescited by 1
  6. 2024Random Aggregate Beamforming for Over-the-Air Federated Learning in Large-Scale Networkscited by 0
  7. 2026Interleaved Noise Injection Improves Clean, Corrupted, and OOD Performancecited by 0