# Rethinking Feature Uncertainty in Stochastic Neural Networks for Adversarial Robustness

arXiv: https://arxiv.org/abs/2201.00148 · HTML: https://mlsanity.com/paper/2201.00148/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 29 references that are in the map.
- First posted on arXiv 1 Jan 2022.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks](https://mlsanity.com/paper/2003.01690/) (2020)
- [Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples](https://mlsanity.com/paper/1802.00420/) (2018)
- [Adversarial Machine Learning at Scale](https://mlsanity.com/paper/1611.01236/) (2016)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [Practical Black-Box Attacks against Machine Learning](https://mlsanity.com/paper/1602.02697/) (2016)
- [ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models](https://mlsanity.com/paper/1708.03999/) (2017)
- [One pixel attack for fooling deep neural networks](https://mlsanity.com/paper/1710.08864/) (2017)
- [Certified Robustness to Adversarial Examples with Differential Privacy](https://mlsanity.com/paper/1802.03471/) (2018)
- [Feature Denoising for Improving Adversarial Robustness](https://mlsanity.com/paper/1812.03411/) (2018)
- [Towards Robust Neural Networks via Random Self-ensemble](https://mlsanity.com/paper/1712.00673/) (2017)
- [Parametric Noise Injection: Trainable Randomness to Improve Deep Neural Network Robustness against Adversarial Attack](https://mlsanity.com/paper/1811.09310/) (2018)
- [Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Network](https://mlsanity.com/paper/1810.01279/) (2018)
- [Adversarial Defense by Restricting the Hidden Space of Deep Neural Networks](https://mlsanity.com/paper/1904.00887/) (2019)
- [Mixup Inference: Better Exploiting Mixup to Defend Adversarial Attacks](https://mlsanity.com/paper/1909.11515/) (2019)
- [Single-step Adversarial training with Dropout Scheduling](https://mlsanity.com/paper/2004.08628/) (2020)
- [Theoretical evidence for adversarial robustness through randomization](https://mlsanity.com/paper/1902.01148/) (2019)
- [Robust Local Features for Improving the Generalization of Adversarial Training](https://mlsanity.com/paper/1909.10147/) (2019)
- [Learn2Perturb: an End-to-end Feature Perturbation Learning to Improve Adversarial Robustness](https://mlsanity.com/paper/2003.01090/) (2020)
- [Towards Achieving Adversarial Robustness by Enforcing Feature Consistency Across Bit Planes](https://mlsanity.com/paper/2004.00306/) (2020)
- [Weight-Covariance Alignment for Adversarially Robust Neural Networks](https://mlsanity.com/paper/2010.08852/) (2020)

## Built on by (most cited first, 2 of 2)

- [Gradient Obfuscation Checklist Test Gives a False Sense of Security](https://mlsanity.com/paper/2206.01705/) (2022)
- [Unveiling the Role of Randomization in Multiclass Adversarial Classification: Insights from Graph Theory](https://mlsanity.com/paper/2503.14299/) (2025)

## Abstract (arXiv)

It is well-known that deep neural networks (DNNs) have shown remarkable success in many fields. However, when adding an imperceptible magnitude perturbation on the model input, the model performance might get rapid decrease. To address this issue, a randomness technique has been proposed recently, named Stochastic Neural Networks (SNNs). Specifically, SNNs inject randomness into the model to defend against unseen attacks and improve the adversarial robustness. However, existed studies on SNNs mainly focus on injecting fixed or learnable noises to model weights/activations. In this paper, we find that the existed SNNs performances are largely bottlenecked by the feature representation ability. Surprisingly, simply maximizing the variance per dimension of the feature distribution leads to a considerable boost beyond all previous methods, which we named maximize feature distribution variance stochastic neural network (MFDV-SNN). Extensive experiments on well-known white- and black-box attacks show that MFDV-SNN achieves a significant improvement over existing methods, which indicates that it is a simple but effective method to improve model robustness.