mlsanityData to 7 Oct 2026

Generalization Bounds for Adversarial Contrastive Learning

arXiv 2302.10633 ↗First posted 21 Feb 2023cs.LG stat.MLcited by 6

Not on the map. Why?

Abstract (arXiv)

Deep networks are well-known to be fragile to adversarial attacks, and adversarial training is one of the most popular methods used to train a robust model. To take advantage of unlabeled data, recent works have applied adversarial training to contrastive learning (Adversarial Contrastive Learning; ACL for short) and obtain promising robust performance. However, the theory of ACL is not well understood. To fill this gap, we leverage the Rademacher complexity to analyze the generalization performance of ACL, with a particular focus on linear models and multi-layer neural networks under ℓp attack (p ≥ 1). Our theory shows that the average adversarial risk of the downstream tasks can be upper bounded by the adversarial unsupervised risk of the upstream task. The experimental results validate our theory.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 23 of its 34 references that are in the map. First posted on arXiv 21 Feb 2023.

Builds on

23 of 34 references.

References · 23

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2013Intriguing properties of neural networkscited by 5,114
  6. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  7. 2016Adversarial examples in the physical worldcited by 1,762
  8. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  9. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  10. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  11. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  12. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,315
11 more
  1. 2019A Theoretical Analysis of Contrastive Unsupervised Representation Learningcited by 530
  2. 2018Adversarially Robust Generalization Requires More Datacited by 318
  3. 2015Norm-Based Capacity Control in Neural Networkscited by 299
  4. 2020Robust Pre-Training by Adversarial Contrastive Learningcited by 144
  5. 2020Adversarial Self-Supervised Contrastive Learningcited by 138
  6. 2018Rademacher Complexity for Adversarially Robust Generalizationcited by 132
  7. 2020Contrastive Learning with Adversarial Examplescited by 91
  8. 2019VC Classes are Adversarially Robustly Learnable, but Only Improperlycited by 83
  9. 2020Adversarial Learning Guarantees for Linear Hypotheses and Neural Networkscited by 47
  10. 2023WAT: Improve the Worst-class Robustness in Adversarial Trainingcited by 27
  11. 2019PAC-Bayesian Contrastive Unsupervised Representation Learningcited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 2; 2026: 1.

6 papers.

  1. 2023Towards Understanding the Mechanism of Contrastive Learning via Similarity Structure: A Theoretical Analysiscited by 6
  2. 2024Understanding Model Ensemble in Transferable Adversarial Attackcited by 6
  3. 2024The Art of Deception: Robust Backdoor Attack using Dynamic Stacking of Triggerscited by 3
  4. 2025Generalization Analysis for Supervised Contrastive Representation Learning under Non-IID Settingscited by 3
  5. 2025How does Labeling Error Impact Contrastive Learning? A Perspective from Data Dimensionality Reductioncited by 1
  6. 2026Optimal VC Dimension of Contrastive Learning with Margincited by 0