mlsanityData to 7 Oct 2026

Enhancing Robust Representation in Adversarial Training: Alignment and Exclusion Criteria

arXiv 2310.03358 ↗First posted 5 Oct 2023cs.CV cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Deep neural networks are vulnerable to adversarial noise. Adversarial Training (AT) has been demonstrated to be the most effective defense strategy to protect neural networks from being fooled. However, we find AT omits to learning robust features, resulting in poor performance of adversarial robustness. To address this issue, we highlight two criteria of robust representation: (1) Exclusion: the feature of examples keeps away from that of other classes; (2) Alignment: the feature of natural and corresponding adversarial examples is close to each other. These motivate us to propose a generic framework of AT to gain robust representation, by the asymmetric negative contrast and reverse attention. Specifically, we design an asymmetric negative contrast based on predicted probabilities, to push away examples of different classes in the feature space. Moreover, we propose to weight feature by parameters of the linear classifier as the reverse attention, to obtain class-aware feature and pull close the feature of the same class. Empirical evaluations on three benchmark datasets show our methods greatly advance the robustness of AT and achieve state-of-the-art performance.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 44 of its 61 references that are in the map. First posted on arXiv 5 Oct 2023.

Builds on

44 of 61 references.

References · 44

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  6. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  8. 2016Wide Residual Networkscited by 3,057
  9. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  10. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  11. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  12. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
32 more
  1. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  2. 2017Defense against Adversarial Attacks Using High-Level Representation Guided Denoisercited by 329
  3. 2017Adversarial Examples for Semantic Segmentation and Object Detectioncited by 275
  4. 2020Attacks Which Do Not Kill Training Make Adversarial Learning Strongercited by 242
  5. 2018Adversarial Logit Pairingcited by 224
  6. 2018MMA Training: Direct Input Space Margin Maximization through Adversarial Trainingcited by 171
  7. 2020Robust Pre-Training by Adversarial Contrastive Learningcited by 144
  8. 2020Adversarial Self-Supervised Contrastive Learningcited by 138
  9. 2020Self-Adaptive Training: beyond Empirical Risk Minimizationcited by 126
  10. 2020Boosting Contrastive Self-Supervised Learning with False Negative Cancellationcited by 119
  11. 2021Robust Learning Meets Generative Models: Can Proxy Distributions Improve Adversarial Robustness?cited by 113
  12. 2021Improving Adversarial Robustness via Channel-wise Activation Suppressingcited by 76
  13. 2021When Does Contrastive Learning Preserve Adversarial Robustness from Pretraining to Finetuning?cited by 76
  14. 2021Bad Characters: Imperceptible NLP Attackscited by 69
  15. 2021Exploring Memorization in Adversarial Trainingcited by 55
  16. 2021Evaluating the Robustness of Semantic Segmentation for Autonomous Driving against Real-World Adversarial Patch Attackscited by 55
  17. 2019Enhancing Adversarial Defense by k-Winners-Take-Allcited by 51
  18. 2022Efficient and Effective Augmentation Strategy for Adversarial Trainingcited by 43
  19. 2021Robust and Accurate Object Detection via Adversarial Learningcited by 42
  20. 2022Enhancing Adversarial Training with Second-Order Statistics of Weightscited by 42
  21. 2021TnT Attacks! Universal Naturalistic Adversarial Patches Against Deep Neural Network Systemscited by 36
  22. 2019Towards Robust Image Classification Using Sequential Attention Modelscited by 27
  23. 2021CIFS: Improving Adversarial Robustness of CNNs via Channel-wise Importance-based Feature Selectioncited by 21
  24. 2022Scaling Adversarial Training to Large Perturbation Boundscited by 20
  25. 2023Query-Efficient Decision-based Black-Box Patch Attackcited by 19
  26. 2019Walking on the Edge: Fast, Low-Distortion Adversarial Examplescited by 18
  27. 2021Mutual Adversarial Training: Learning together is better than going alonecited by 17
  28. 2023Efficient Adversarial Contrastive Learning via Robustness-Aware Coreset Selectioncited by 14
  29. 2022Adversarial Contrastive Learning via Asymmetric InfoNCEcited by 13
  30. 2019A Robust Approach for Securing Audio Classification Against Adversarial Attackscited by 12
  31. 2021Modeling Adversarial Noise for Adversarial Trainingcited by 8
  32. 2023Robust Proxy: Improving Adversarial Robustness by Robust Proxy Learningcited by 3

Built on by

Not cited yet.