# Wavelets Beat Monkeys at Adversarial Robustness

arXiv: https://arxiv.org/abs/2304.09403 · HTML: https://mlsanity.com/paper/2304.09403/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 44 references that are in the map.
- First posted on arXiv 19 Apr 2023.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks](https://mlsanity.com/paper/2003.01690/) (2020)
- [Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples](https://mlsanity.com/paper/1802.00420/) (2018)
- [Adversarial Examples Are Not Bugs, They Are Features](https://mlsanity.com/paper/1905.02175/) (2019)
- [Robustness May Be at Odds with Accuracy](https://mlsanity.com/paper/1805.12152/) (2018)
- [Fast is better than free: Revisiting adversarial training](https://mlsanity.com/paper/2001.03994/) (2020)
- [Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks](https://mlsanity.com/paper/1511.04508/) (2015)
- [Adversarial Training for Free!](https://mlsanity.com/paper/1904.12843/) (2019)
- [One pixel attack for fooling deep neural networks](https://mlsanity.com/paper/1710.08864/) (2017)
- [Differentially Private Learning Needs Better Features (or Much More Data)](https://mlsanity.com/paper/2011.11660/) (2020)
- [Partial success in closing the gap between human and machine vision](https://mlsanity.com/paper/2106.07411/) (2021)
- [Boosting Adversarial Training with Hypersphere Embedding](https://mlsanity.com/paper/2002.08619/) (2020)
- [Brain-Like Object Recognition with High-Performing Shallow Recurrent ANNs](https://mlsanity.com/paper/1909.06161/) (2019)
- [Kymatio: Scattering Transforms in Python](https://mlsanity.com/paper/1812.11214/) (2018)
- [Robust Encodings: A Framework for Combating Adversarial Typos](https://mlsanity.com/paper/2005.01229/) (2020)
- [Deep Roto-Translation Scattering for Object Classification](https://mlsanity.com/paper/1412.8659/) (2014)
- [Solid Harmonic Wavelet Scattering for Predictions of Molecule Properties](https://mlsanity.com/paper/1805.00571/) (2018)
- [Intermittent process analysis with scattering moments](https://mlsanity.com/paper/1311.4104/) (2013)
- [Wavelet Scattering Regression of Quantum Chemical Energies](https://mlsanity.com/paper/1605.04654/) (2016)
- [Adversarially Robust Low Dimensional Representations](https://mlsanity.com/paper/1911.13268/) (2019)
- [How to quantify fields or textures? A guide to the scattering transform](https://mlsanity.com/paper/2112.01288/) (2021)
- [A Spectral View of Adversarially Robust Features](https://mlsanity.com/paper/1811.06609/) (2018)
- [Wavelet Scattering Networks for Atomistic Systems with Extrapolation of Material Properties](https://mlsanity.com/paper/2006.01247/) (2020)
- [Quantification of MagLIF morphology using the Mallat Scattering Transformation](https://mlsanity.com/paper/2005.01600/) (2020)
- [Three-Dimensional Fourier Scattering Transform and Classification of Hyperspectral Images](https://mlsanity.com/paper/1906.06804/) (2019)

## Built on by (most cited first, 2 of 2)

- [Energy Propagation in Scattering Convolution Networks Can Be Arbitrarily Slow](https://mlsanity.com/paper/2406.05121/) (2024)
- [Classification-Denoising Networks](https://mlsanity.com/paper/2410.03505/) (2024)

## Abstract (arXiv)

Research on improving the robustness of neural networks to adversarial noise - imperceptible malicious perturbations of the data - has received significant attention. The currently uncontested state-of-the-art defense to obtain robust deep neural networks is Adversarial Training (AT), but it consumes significantly more resources compared to standard training and trades off accuracy for robustness. An inspiring recent work [Dapello et al.] aims to bring neurobiological tools to the question: How can we develop Neural Nets that robustly generalize like human vision? [Dapello et al.] design a network structure with a neural hidden first layer that mimics the primate primary visual cortex (V1), followed by a back-end structure adapted from current CNN vision models. It seems to achieve non-trivial adversarial robustness on standard vision benchmarks when tested on small perturbations. Here we revisit this biologically inspired work, and ask whether a principled parameter-free representation with inspiration from physics is able to achieve the same goal. We discover that the wavelet scattering transform can replace the complex V1-cortex and simple uniform Gaussian noise can take the role of neural stochasticity, to achieve adversarial robustness. In extensive experiments on the CIFAR-10 benchmark with adaptive adversarial attacks we show that: 1) Robustness of VOneBlock architectures is relatively weak (though non-zero) when the strength of the adversarial attack radius is set to commonly used benchmarks. 2) Replacing the front-end VOneBlock by an off-the-shelf parameter-free Scatternet followed by simple uniform Gaussian noise can achieve much more substantial adversarial robustness without adversarial training. Our work shows how physically inspired structures yield new insights into robustness that were previously only thought possible by meticulously mimicking the human cortex.