mlsanityData to 7 Oct 2026

Rethinking Robust Contrastive Learning from the Adversarial Perspective

arXiv 2302.02502 ↗First posted 5 Feb 2023cs.LG cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

To advance the understanding of robust deep learning, we delve into the effects of adversarial training on self-supervised and supervised contrastive learning alongside supervised learning. Our analysis uncovers significant disparities between adversarial and clean representations in standard-trained networks across various learning algorithms. Remarkably, adversarial training mitigates these disparities and fosters the convergence of representations toward a universal set, regardless of the learning scheme used. Additionally, increasing the similarity between adversarial and clean representations, particularly near the end of the network, enhances network robustness. These findings offer valuable insights for designing and training effective and robust deep learning networks. Our code is released at textcolormagentaurlhttps://github.com/softsys4ai/CL-Robustness.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 22 of its 27 references that are in the map. First posted on arXiv 5 Feb 2023.

Builds on

22 of 27 references.

References · 22

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  5. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  6. 2020Supervised Contrastive Learningcited by 2,803
  7. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  8. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  9. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  10. 2019Using Self-Supervised Learning Can Improve Model Robustness and Uncertaintycited by 511
  11. 2020Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depthcited by 241
  12. 2020Representation Learning via Invariant Causal Mechanismscited by 197
10 more
  1. 2020Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuningcited by 152
  2. 2020Robust Pre-Training by Adversarial Contrastive Learningcited by 144
  3. 2021A Broad Study on the Transferability of Visual Representations with Contrastive Learningcited by 78
  4. 2021When Does Contrastive Learning Preserve Adversarial Robustness from Pretraining to Finetuning?cited by 76
  5. 2019Adversarially Robust Generalization Just Requires More Unlabeled Datacited by 75
  6. 2020A Causal View on Robustness of Neural Networkscited by 53
  7. 2022Is Self-Supervised Learning More Robust Than Supervised Learning?cited by 17
  8. 2022Understanding Robust Learning through the Lens of Representation Similaritiescited by 12
  9. 2020Adversarial defense for deep speaker recognition using hybrid adversarial trainingcited by 5
  10. 2022Adversarial Contrastive Learning by Permuting Cluster Assignmentscited by 2

Built on by

Not cited yet.