Wavelet Regularization Benefits Adversarial Training
Not on the map. Why?
Abstract (arXiv)
Adversarial training methods are state-of-the-art (SOTA) empirical defense methods against adversarial examples. Many regularization methods have been proven to be effective with the combination of adversarial training. Nevertheless, such regularization methods are implemented in the time domain. Since adversarial vulnerability can be regarded as a high-frequency phenomenon, it is essential to regulate the adversarially-trained neural network models in the frequency domain. Faced with these challenges, we make a theoretical analysis on the regularization property of wavelets which can enhance adversarial training. We propose a wavelet regularization method based on the Haar wavelet decomposition which is named Wavelet Average Pooling. This wavelet regularization module is integrated into the wide residual neural network so that a new WideWaveletResNet model is formed. On the datasets of CIFAR-10 and CIFAR-100, our proposed Adversarial Wavelet Training method realizes considerable robustness under different types of attacks. It verifies the assumption that our wavelet regularization method can enhance adversarial robustness especially in the deep wide neural networks. The visualization experiments of the Frequency Principle (F-Principle) and interpretability are implemented to show the effectiveness of our method. A detailed comparison based on different wavelet base functions is presented. The code is available at the repository: urlhttps://github.com/momo1986/AdversarialWaveletTraining.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 80 of its 104 references that are in the map. First posted on arXiv 8 Jun 2022.
Builds on
80 of 104 references.
References · 80
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2014Explaining and Harnessing Adversarial Examplescited by 7,364
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
- 2013Intriguing properties of neural networkscited by 5,155
- 2016Wide Residual Networkscited by 3,070
- 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
- 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
- 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
- 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
- 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,318
- 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,268
38 more
- 2018Robustness May Be at Odds with Accuracycited by 880
- 2017Ensemble Adversarial Training: Attacks and Defensescited by 840
- 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 787
- 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 738
- 2020Fast is better than free: Revisiting adversarial trainingcited by 727
- 2014Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognitioncited by 720
- 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 678
- 2017ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Modelscited by 666
- 2019Adversarial Training for Free!cited by 643
- 2018Gradient Descent Finds Global Minima of Deep Neural Networkscited by 574
- 2020On Adaptive Attacks to Adversarial Example Defensescited by 555
- 2017Countering Adversarial Images using Input Transformationscited by 520
- 2020Overfitting in adversarially robust deep learningcited by 506
- 2018Black-box Adversarial Attacks with Limited Queries and Informationcited by 501
- 2017Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networkscited by 486
- 2017Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Modelscited by 462
- 2019Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attackscited by 442
- 2017Mitigating Adversarial Effects Through Randomizationcited by 383
- 2019Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networkscited by 368
- 2017Defense against Adversarial Attacks Using High-Level Representation Guided Denoisercited by 329
- 2018Gaussian Process Behaviour in Wide Deep Neural Networkscited by 317
- 2019High Frequency Component Helps Explain the Generalization of Convolutional Neural Networkscited by 310
- 2019A Fourier Perspective on Model Robustness in Computer Visioncited by 288
- 2018Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionalitycited by 282
- 2017PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examplescited by 244
- 2020Attacks Which Do Not Kill Training Make Adversarial Learning Strongercited by 243
- 2017Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradientscited by 236
- 2018Adversarial Logit Pairingcited by 224
- 2019Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networkscited by 222
- 2019Improving Adversarial Robustness via Promoting Ensemble Diversitycited by 213
- 2020Bag of Tricks for Adversarial Trainingcited by 200
- 2021Data Augmentation Can Improve Robustnesscited by 194
- 2019Towards moderate overparameterization: global convergence guarantees for training shallow neural networkscited by 176
- 2018Robustness via curvature regularization, and vice versacited by 175
- 2017Towards Robust Neural Networks via Random Self-ensemblecited by 159
- 2021A Universal Law of Robustness via Isoperimetrycited by 151
- 2020Understanding and Mitigating the Tradeoff Between Robustness and Accuracycited by 149
- 2018Prior Convictions: Black-Box Adversarial Attacks with Bandits and Priorscited by 146
50 most cited of 80 All 80 (JSON)
Built on by
Citing papers per year: 2023: 1; 2024: 0; 2025: 2.
3 papers.