Topology-preserving Adversarial Training for Alleviating Natural Accuracy Degradation
Not on the map. Why?
Abstract (arXiv)
Despite the effectiveness in improving the robustness of neural networks, adversarial training has suffered from the natural accuracy degradation problem, i.e., accuracy on natural samples has reduced significantly. In this study, we reveal that natural accuracy degradation is highly related to the disruption of the natural sample topology in the representation space by quantitative and qualitative experiments. Based on this observation, we propose Topology-pReserving Adversarial traINing (TRAIN) to alleviate the problem by preserving the topology structure of natural samples from a standard model trained only on natural samples during adversarial training. As an additional regularization, our method can be combined with various popular adversarial training algorithms, taking advantage of both sides. Extensive experiments on CIFAR-10, CIFAR-100, and Tiny ImageNet show that our proposed method achieves consistent and significant improvements over various strong baselines in most cases. Specifically, without additional data, TRAIN achieves up to 8.86% improvement in natural accuracy and 6.33% improvement in robust accuracy.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 48 of its 64 references that are in the map. First posted on arXiv 29 Nov 2023.
Builds on
48 of 64 references.
References · 48
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
- 2013Intriguing properties of neural networkscited by 5,155
- 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
- 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
- 2014FitNets: Hints for Thin Deep Netscited by 1,653
- 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
- 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
- 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,268
- 2019Relational Knowledge Distillationcited by 889
- 2019Contrastive Representation Distillationcited by 748
- 2020Fast is better than free: Revisiting adversarial trainingcited by 727
36 more
- 2020On Adaptive Attacks to Adversarial Example Defensescited by 555
- 2019Similarity-Preserving Knowledge Distillationcited by 516
- 2017Mitigating Adversarial Effects Through Randomizationcited by 383
- 2021Distilling Knowledge via Knowledge Reviewcited by 288
- 2021Recent Advances in Adversarial Training for Adversarial Robustnesscited by 279
- 2020Attacks Which Do Not Kill Training Make Adversarial Learning Strongercited by 243
- 2018Adversarial Logit Pairingcited by 224
- 2018Learning Deep Representations with Probabilistic Knowledge Transfercited by 213
- 2023Better Diffusion Models Further Improve Adversarial Trainingcited by 208
- 2020Bag of Tricks for Adversarial Trainingcited by 200
- 2020Geometry-aware Instance-reweighted Adversarial Trainingcited by 198
- 2021Data Augmentation Can Improve Robustnesscited by 194
- 2021Few-Shot Incremental Learning with Continually Evolved Classifierscited by 187
- 2018Stochastic Activation Pruning for Robust Adversarial Defensecited by 171
- 2019Adversarially Robust Distillationcited by 125
- 2022Robustness and Accuracy Could Be Reconcilable by (Proper) Definitioncited by 111
- 2019Soft-Label Dataset Distillation and Text Dataset Distillationcited by 107
- 2018Towards the first adversarially robust neural network model on MNISTcited by 104
- 2022LAS-AT: Adversarial Training with Learnable Attack Strategycited by 95
- 2020Learnable Boundary Guided Adversarial Trainingcited by 93
- 2019Adversarial Robustness Against the Union of Multiple Perturbation Modelscited by 88
- 2021Revisiting Adversarial Robustness Distillation: Robust Soft Labels Make Student Bettercited by 79
- 2023Revisiting Adversarial Training for ImageNet: Architectures, Training and Generalization across Threat Modelscited by 79
- 2019Metric Learning for Adversarial Robustnesscited by 77
- 2021Complementary Relation Contrastive Distillationcited by 42
- 2021Subspace Adversarial Trainingcited by 41
- 2019Knowledge Flow: Improve Upon Your Teacherscited by 36
- 2020SAT: Improving Adversarial Training via Curriculum-Based Loss Smoothingcited by 26
- 2020Adversarial Feature Selection against Evasion Attackscited by 13
- 2021LTD: Low Temperature Distillation for Gradient Masking-free Adversarial Trainingcited by 13
- 2020Adversarial Feature Desensitizationcited by 12
- 2020Adversarial Concurrent Training: Optimizing Robustness and Accuracy Trade-off of Deep Neural Networkscited by 9
- 2021Improving Model Robustness with Latent Distribution Locally and Globallycited by 9
- 2021ε-weakened Robustness of Deep Neural Networkscited by 8
- 2021Adversarial Learning with Cost-Sensitive Classescited by 6
- 2021Get Fooled for the Right Reason: Improving Adversarial Robustness through a Teacher-guided Curriculum Learning Approachcited by 4
Built on by
Citing papers per year: 2024: 1.
1 paper.