mlsanityData to 7 Oct 2026

Understanding and Achieving Efficient Robustness with Adversarial Supervised Contrastive Learning

arXiv 2101.10027 ↗First posted 25 Jan 2021cs.LG cs.AI cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

Contrastive learning (CL) has recently emerged as an effective approach to learning representation in a range of downstream tasks. Central to this approach is the selection of positive (similar) and negative (dissimilar) sets to provide the model the opportunity to `contrast' between data and class representation in the latent space. In this paper, we investigate CL for improving model robustness using adversarial samples. We first designed and performed a comprehensive study to understand how adversarial vulnerability behaves in the latent space. Based on this empirical evidence, we propose an effective and efficient supervised contrastive learning to achieve model robustness against adversarial attacks. Moreover, we propose a new sample selection strategy that optimizes the positive/negative sets by removing redundancy and improving correlation with the anchor. Extensive experiments show that our Adversarial Supervised Contrastive Learning (ASCL) approach achieves comparable performance with the state-of-the-art defenses while significantly outperforms other CL-based defense methods by using only 42.8% positives and 6.3% negatives.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 42 of its 55 references that are in the map. First posted on arXiv 25 Jan 2021.

Builds on

42 of 55 references.

References · 42

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  6. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  7. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  8. 2016Wide Residual Networkscited by 3,057
  9. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  10. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  11. 2020Supervised Contrastive Learningcited by 2,803
  12. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
30 more
  1. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  2. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  3. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  4. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 918
  5. 2020What Makes for Good Views for Contrastive Learning?cited by 843
  6. 2015Deep Learning and the Information Bottleneck Principlecited by 793
  7. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  8. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 682
  9. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 659
  10. 2019Adversarial Training for Free!cited by 638
  11. 2018Certified Robustness to Adversarial Examples with Differential Privacycited by 462
  12. 2018Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Surveycited by 459
  13. 2019On Evaluating Adversarial Robustnesscited by 442
  14. 2018Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Modelscited by 362
  15. 2019Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attackcited by 333
  16. 2018Feature Denoising for Improving Adversarial Robustnesscited by 312
  17. 2020Attacks Which Do Not Kill Training Make Adversarial Learning Strongercited by 242
  18. 2017On Detecting Adversarial Perturbationscited by 232
  19. 2019Improving Adversarial Robustness via Promoting Ensemble Diversitycited by 213
  20. 2020Bag of Tricks for Adversarial Trainingcited by 199
  21. 2020Robust Pre-Training by Adversarial Contrastive Learningcited by 144
  22. 2020Adversarial Self-Supervised Contrastive Learningcited by 138
  23. 2020Self-Adaptive Training: beyond Empirical Risk Minimizationcited by 126
  24. 2019Defense Against Adversarial Attacks Using Feature Scattering-based Adversarial Trainingcited by 95
  25. 2020Learnable Boundary Guided Adversarial Trainingcited by 93
  26. 2020Smooth Adversarial Trainingcited by 89
  27. 2019Metric Learning for Adversarial Robustnesscited by 77
  28. 2020Are all negatives created equal in contrastive instance discrimination?cited by 50
  29. 2019Scaling up the randomized gradient-free adversarial attack reveals overestimation of robustness using established attackscited by 11
  30. 2020Improving Adversarial Robustness by Enforcing Local and Global Compactnesscited by 11

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 2; 2023: 2; 2024: 2; 2025: 1.

8 papers.

  1. 2022A Unified Wasserstein Distributional Robustness Framework for Adversarial Trainingcited by 36
  2. 2021Contrastive Learning for Fair Representationscited by 34
  3. 2025Fantastic Targets for Concept Erasure in Diffusion Models and Where To Find Themcited by 34
  4. 2024Adversarial Feature Alignment: Balancing Robustness and Accuracy in Deep Learning via Adversarial Trainingcited by 5
  5. 2023Generating Adversarial Examples with Task Oriented Multi-Objective Optimizationcited by 2
  6. 2024TIMA: Text-Image Mutual Awareness for Balancing Zero-Shot Adversarial Robustness and Generalization Abilitycited by 2
  7. 2022ARIA: Adversarially Robust Image Attribution for Content Provenancecited by 1
  8. 2023Generalization Bounds for Robust Contrastive Learning: From Theory to Practicecited by 0