# Weight-Covariance Alignment for Adversarially Robust Neural Networks

arXiv: https://arxiv.org/abs/2010.08852 · HTML: https://mlsanity.com/paper/2010.08852/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Oct 2020.

## Built on by (most cited first, 9 of 9)

- [GradDiv: Adversarial Robustness of Randomized Neural Networks via Gradient Diversity Regularization](https://mlsanity.com/paper/2107.02425/) (2021)
- [Exploring Novel Pooling Strategies for Edge Preserved Feature Maps in Convolutional Neural Networks](https://mlsanity.com/paper/2110.08842/) (2021)
- [Rethinking Feature Uncertainty in Stochastic Neural Networks for Adversarial Robustness](https://mlsanity.com/paper/2201.00148/) (2022)
- [Attacking Adversarial Defences by Smoothing the Loss Landscape](https://mlsanity.com/paper/2208.00862/) (2022)
- [Deciphering the Definition of Adversarial Robustness for post-hoc OOD Detectors](https://mlsanity.com/paper/2406.15104/) (2024)
- [How Sampling Impacts the Robustness of Stochastic Neural Networks](https://mlsanity.com/paper/2204.10839/) (2022)
- [Gradient Obfuscation Checklist Test Gives a False Sense of Security](https://mlsanity.com/paper/2206.01705/) (2022)
- [Adversarial Augmentation Training Makes Action Recognition Models More Robust to Realistic Video Distribution Shifts](https://mlsanity.com/paper/2401.11406/) (2024)
- [Feature Statistics with Uncertainty Help Adversarial Robustness](https://mlsanity.com/paper/2503.20583/) (2025)

## Abstract (arXiv)

Stochastic Neural Networks (SNNs) that inject noise into their hidden layers have recently been shown to achieve strong robustness against adversarial attacks. However, existing SNNs are usually heuristically motivated, and often rely on adversarial training, which is computationally costly. We propose a new SNN that achieves state-of-the-art performance without relying on adversarial training, and enjoys solid theoretical justification. Specifically, while existing SNNs inject learned or hand-tuned isotropic noise, our SNN learns an anisotropic noise distribution to optimize a learning-theoretic bound on adversarial robustness. We evaluate our method on a number of popular benchmarks, show that it can be applied to different architectures, and that it provides robustness to a variety of white-box and black-box attacks, while being simple and fast to train compared to existing alternatives.