# Contrastive Learning with Adversarial Examples

arXiv: https://arxiv.org/abs/2010.12050 · HTML: https://mlsanity.com/paper/2010.12050/

## Key facts

- Cited by 91 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Oct 2020.

## Built on by (most cited first, 30 of 91)

- [SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation](https://mlsanity.com/paper/2202.03104/) (2022)
- [On Feature Decorrelation in Self-Supervised Learning](https://mlsanity.com/paper/2105.00470/) (2021)
- [Advances in adversarial attacks and defenses in computer vision: A survey](https://mlsanity.com/paper/2108.00401/) (2021)
- [Can contrastive learning avoid shortcut solutions?](https://mlsanity.com/paper/2106.11230/) (2021)
- [Rethinking Minimal Sufficient Representation in Contrastive Learning](https://mlsanity.com/paper/2203.07004/) (2022)
- [Unpaired Deep Image Deraining Using Dual Contrastive Learning](https://mlsanity.com/paper/2109.02973/) (2021)
- [Dual Temperature Helps Contrastive Learning Without Many Negative Samples: Towards Understanding and Simplifying MoCo](https://mlsanity.com/paper/2203.17248/) (2022)
- [Improving Transferability of Representations via Augmentation-Aware Self-Supervision](https://mlsanity.com/paper/2111.09613/) (2021)
- [Contrastive Learning for Neural Topic Model](https://mlsanity.com/paper/2110.12764/) (2021)
- [Contrastive Multimodal Fusion with TupleInfoNCE](https://mlsanity.com/paper/2107.02575/) (2021)
- [Robust Contrastive Learning Using Negative Samples with Diminished Semantics](https://mlsanity.com/paper/2110.14189/) (2021)
- [Adversarial Graph Contrastive Learning with Information Regularization](https://mlsanity.com/paper/2202.06491/) (2022)
- [Exploring the Impact of Negative Samples of Contrastive Learning: A Case Study of Sentence Embedding](https://mlsanity.com/paper/2202.13093/) (2022)
- [Data Augmentation of Contrastive Learning is Estimating Positive-incentive Noise](https://mlsanity.com/paper/2408.09929/) (2024)
- [Federated Contrastive Learning for Dermatological Disease Diagnosis via On-device Learning](https://mlsanity.com/paper/2202.07470/) (2022)
- [Exploring Robustness of Unsupervised Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/2105.10843/) (2021)
- [Rethinking the Effect of Data Augmentation in Adversarial Contrastive Learning](https://mlsanity.com/paper/2303.01289/) (2023)
- [Finding Order in Chaos: A Novel Data Augmentation Method for Time Series in Contrastive Learning](https://mlsanity.com/paper/2309.13439/) (2023)
- [Adversarial Training of Self-supervised Monocular Depth Estimation against Physical-World Attacks](https://mlsanity.com/paper/2301.13487/) (2023)
- [Towards Robust Graph Contrastive Learning](https://mlsanity.com/paper/2102.13085/) (2021)
- [Adversarial Examples can be Effective Data Augmentation for Unsupervised Machine Learning](https://mlsanity.com/paper/2103.01895/) (2021)
- [Does Negative Sampling Matter? A Review with Insights into its Theory and Applications](https://mlsanity.com/paper/2402.17238/) (2024)
- [Efficient Adversarial Contrastive Learning via Robustness-Aware Coreset Selection](https://mlsanity.com/paper/2302.03857/) (2023)
- [A Unified Contrastive Energy-based Model for Understanding the Generative Ability of Adversarial Training](https://mlsanity.com/paper/2203.13455/) (2022)
- [RODD: A Self-Supervised Approach for Robust Out-of-Distribution Detection](https://mlsanity.com/paper/2204.02553/) (2022)
- [Synthetic Data Can Also Teach: Synthesizing Effective Data for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/2202.06464/) (2022)
- [Sample Efficient Detection and Classification of Adversarial Attacks via Self-Supervised Embeddings](https://mlsanity.com/paper/2108.13797/) (2021)
- [Revisiting Contrastive Learning through the Lens of Neighborhood Component Analysis: an Integrated Framework](https://mlsanity.com/paper/2112.04468/) (2021)
- [Non-generative Generalized Zero-shot Learning via Task-correlated Disentanglement and Controllable Samples Synthesis](https://mlsanity.com/paper/2203.05335/) (2022)
- [FuseFL: One-Shot Federated Learning through the Lens of Causality with Progressive Model Fusion](https://mlsanity.com/paper/2410.20380/) (2024)

## Abstract (arXiv)

Contrastive learning (CL) is a popular technique for self-supervised learning (SSL) of visual representations. It uses pairs of augmentations of unlabeled training examples to define a classification task for pretext learning of a deep embedding. Despite extensive works in augmentation procedures, prior works do not address the selection of challenging negative pairs, as images within a sampled batch are treated independently. This paper addresses the problem, by introducing a new family of adversarial examples for constrastive learning and using these examples to define a new adversarial training algorithm for SSL, denoted as CLAE. When compared to standard CL, the use of adversarial examples creates more challenging positive pairs and adversarial training produces harder negative pairs by accounting for all images in a batch during the optimization. CLAE is compatible with many CL methods in the literature. Experiments show that it improves the performance of several existing CL baselines on multiple datasets.