# Fixed Inter-Neuron Covariability Induces Adversarial Robustness

arXiv: https://arxiv.org/abs/2308.03956 · HTML: https://mlsanity.com/paper/2308.03956/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 36 references that are in the map.
- First posted on arXiv 7 Aug 2023.

## Builds on (most cited first, 21 of 21)

- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks](https://mlsanity.com/paper/2003.01690/) (2020)
- [Certified Adversarial Robustness via Randomized Smoothing](https://mlsanity.com/paper/1902.02918/) (2019)
- [Adversarial Examples Are Not Bugs, They Are Features](https://mlsanity.com/paper/1905.02175/) (2019)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [Recent Advances in Adversarial Training for Adversarial Robustness](https://mlsanity.com/paper/2102.01356/) (2021)
- [Data Augmentation Can Improve Robustness](https://mlsanity.com/paper/2111.05328/) (2021)
- [Certified Adversarial Robustness with Additive Noise](https://mlsanity.com/paper/1809.03113/) (2018)
- [Towards the first adversarially robust neural network model on MNIST](https://mlsanity.com/paper/1805.09190/) (2018)
- [Certified Defense to Image Transformations via Randomized Smoothing](https://mlsanity.com/paper/2002.12463/) (2020)
- [A scalable noisy speech dataset and online subjective test framework](https://mlsanity.com/paper/1909.08050/) (2019)
- [Dense Associative Memory is Robust to Adversarial Inputs](https://mlsanity.com/paper/1701.00939/) (2017)
- [Foveation-based Mechanisms Alleviate Adversarial Examples](https://mlsanity.com/paper/1511.06292/) (2015)
- [Biologically Inspired Mechanisms for Adversarial Robustness](https://mlsanity.com/paper/2006.16427/) (2020)
- [FoveaTer: Foveated Transformer for Image Classification](https://mlsanity.com/paper/2105.14173/) (2021)
- [Attacking the Madry Defense Model with L1-based Adversarial Examples](https://mlsanity.com/paper/1710.10733/) (2017)
- [Predify: Augmenting deep neural networks with brain-inspired predictive coding dynamics](https://mlsanity.com/paper/2106.02749/) (2021)
- [Center Smoothing: Certified Robustness for Networks with Structured Outputs](https://mlsanity.com/paper/2102.09701/) (2021)

## Abstract (arXiv)

The vulnerability to adversarial perturbations is a major flaw of Deep Neural Networks (DNNs) that raises question about their reliability when in real-world scenarios. On the other hand, human perception, which DNNs are supposed to emulate, is highly robust to such perturbations, indicating that there may be certain features of the human perception that make it robust but are not represented in the current class of DNNs. One such feature is that the activity of biological neurons is correlated and the structure of this correlation tends to be rather rigid over long spans of times, even if it hampers performance and learning. We hypothesize that integrating such constraints on the activations of a DNN would improve its adversarial robustness, and, to test this hypothesis, we have developed the Self-Consistent Activation (SCA) layer, which comprises of neurons whose activations are consistent with each other, as they conform to a fixed, but learned, covariability pattern. When evaluated on image and sound recognition tasks, the models with a SCA layer achieved high accuracy, and exhibited significantly greater robustness than multi-layer perceptron models to state-of-the-art Auto-PGD adversarial attacks textitwithout being trained on adversarially perturbed data