Dual Head Adversarial Training
Not on the map. Why?
Abstract (arXiv)
Deep neural networks (DNNs) are known to be vulnerable to adversarial examples/attacks, raising concerns about their reliability in safety-critical applications. A number of defense methods have been proposed to train robust DNNs resistant to adversarial attacks, among which adversarial training has so far demonstrated the most promising results. However, recent studies have shown that there exists an inherent tradeoff between accuracy and robustness in adversarially-trained DNNs. In this paper, we propose a novel technique Dual Head Adversarial Training (DH-AT) to further improve the robustness of existing adversarial training methods. Different from existing improved variants of adversarial training, DH-AT modifies both the architecture of the network and the training strategy to seek more robustness. Specifically, DH-AT first attaches a second network head (or branch) to one intermediate layer of the network, then uses a lightweight convolutional neural network (CNN) to aggregate the outputs of the two heads. The training strategy is also adapted to reflect the relative importance of the two heads. We empirically show, on multiple benchmark datasets, that DH-AT can bring notable robustness improvements to existing adversarial training methods. Compared with TRADES, one state-of-the-art adversarial training method, our DH-AT can improve the robustness by 3.4% against PGD40 and 2.3% against AutoAttack, and also improve the clean accuracy by 1.8%.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 47 of its 56 references that are in the map. First posted on arXiv 21 Apr 2021.
Builds on
47 of 56 references.
References · 47
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2013Intriguing properties of neural networkscited by 5,114
- 2016Wide Residual Networkscited by 3,057
- 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
- 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
- 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
- 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,315
- 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
- 2016Adversarial Machine Learning at Scalecited by 934
35 more
- 2018Robustness May Be at Odds with Accuracycited by 873
- 2017Ensemble Adversarial Training: Attacks and Defensescited by 833
- 2015The Limitations of Deep Learning in Adversarial Settingscited by 784
- 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 675
- 2017Countering Adversarial Images using Input Transformationscited by 517
- 2020Overfitting in adversarially robust deep learningcited by 504
- 2017Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networkscited by 484
- 2015Deep Speech 2: End-to-End Speech Recognition in English and Mandarincited by 449
- 2017Mitigating Adversarial Effects Through Randomizationcited by 382
- 2018Adversarial Attack on Graph Structured Datacited by 300
- 2019Adversarial Examples Improve Image Recognitioncited by 283
- 2018Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionalitycited by 280
- 2017Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradientscited by 236
- 2020Bag of Tricks for Adversarial Trainingcited by 199
- 2018Stochastic Activation Pruning for Robust Adversarial Defensecited by 171
- 2020Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNetscited by 165
- 2019Are Labels Required for Improving Adversarial Robustness?cited by 162
- 2019Understanding Adversarial Attacks on Deep Learning Based Medical Image Analysis Systemscited by 151
- 2020Physically Realizable Adversarial Examples for LiDAR Object Detectioncited by 137
- 2020Self-Adaptive Training: beyond Empirical Risk Minimizationcited by 126
- 2020Adversarial Camouflage: Hiding Physical-World Attacks with Natural Stylescited by 126
- 2019Adversarial Robustness vs Model Compression, or Both?cited by 98
- 2020Boosting Adversarial Training with Hypersphere Embeddingcited by 97
- 2020Learnable Boundary Guided Adversarial Trainingcited by 93
- 2018Improving DNN Robustness to Adversarial Attacks using Jacobian Regularizationcited by 88
- 2021Improving Adversarial Robustness via Channel-wise Activation Suppressingcited by 76
- 2019PhysGAN: Generating Physical-World-Resilient Adversarial Examples for Autonomous Drivingcited by 72
- 2018Sparse DNNs with Improved Adversarial Robustnesscited by 71
- 2020Adv-BERT: BERT is not robust on misspellings! Generating nature adversarial samples on BERTcited by 69
- 2019Adversarial Objects Against LiDAR-Based Autonomous Driving Systemscited by 62
- 2019Black-box Adversarial Attacks on Video Recognition Modelscited by 58
- 2019Scaleable input gradient regularization for adversarial robustnesscited by 45
- 2018Unravelling Robustness of Deep Learning based Face Recognition Against Adversarial Attackscited by 31
- 2018Distributionally Adversarial Attackcited by 24
- 2020On the Limitations of Denoising Strategies as Adversarial Defensescited by 9
Built on by
Citing papers per year: 2023: 1.
1 paper.