Robustness of Unsupervised Representation Learning without Labels
Not on the map. Why?
Abstract (arXiv)
Unsupervised representation learning leverages large unlabeled datasets and is competitive with supervised learning. But non-robust encoders may affect downstream task robustness. Recently, robust representation encoders have become of interest. Still, all prior work evaluates robustness using a downstream classification task. Instead, we propose a family of unsupervised robustness measures, which are model- and task-agnostic and label-free. We benchmark state-of-the-art representation encoders and show that none dominates the rest. We offer unsupervised extensions to the FGSM and PGD attacks. When used in adversarial training, they improve most unsupervised robustness measures, including certified robustness. We validate our results against a linear probe and show that, for MOCOv2, adversarial training results in 3 times higher certified accuracy, a 2-fold decrease in impersonation attack success rate and considerable improvements in certified robustness.
Key facts. Cited by 2 papers in the map (data through 13 Oct 2026). Builds on 56 of its 70 references that are in the map. First posted on arXiv 8 Oct 2022.
Builds on
56 of 70 references.
References · 56
- 2017Attention Is All You Needcited by 60,106
- 2015Deep Residual Learning for Image Recognitioncited by 51,950
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,223
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,009
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,476
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,712
- 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,492
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,364
- 2014Explaining and Harnessing Adversarial Examplescited by 7,336
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,379
- 2022Generative Adversarial Networkscited by 5,614
- 2013Intriguing properties of neural networkscited by 5,137
38 more
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,730
- 2020Exploring Simple Siamese Representation Learningcited by 2,860
- 2020Improved Baselines with Momentum Contrastive Learningcited by 2,285
- 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,826
- 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,612
- 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,511
- 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,378
- 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,336
- 2019Contrastive Multiview Codingcited by 1,297
- 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,265
- 2016Adversarial Machine Learning at Scalecited by 935
- 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 919
- 2018Robustness May Be at Odds with Accuracycited by 878
- 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 733
- 2017Evasion Attacks against Machine Learning at Test Timecited by 689
- 2019Adversarial Training for Free!cited by 641
- 2014Semi-Supervised Learning with Deep Generative Modelscited by 494
- 2018Certified Robustness to Adversarial Examples with Differential Privacycited by 467
- 2019Unlabeled Data Improves Adversarial Robustnesscited by 416
- 2020Keystroke Dynamics as Part of Lifeloggingcited by 334
- 2019High Frequency Component Helps Explain the Generalization of Convolutional Neural Networkscited by 310
- 2020Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learningcited by 297
- 2021Recent Advances in Adversarial Training for Adversarial Robustnesscited by 278
- 2019Large Scale Adversarial Representation Learningcited by 239
- 2020Large image datasets: A pyrrhic win for computer vision?cited by 237
- 2018Adversarial Risk and the Dangers of Evaluating Against Weak Attackscited by 221
- 2018Formal Limitations on the Measurement of Mutual Informationcited by 189
- 2019Are Labels Required for Improving Adversarial Robustness?cited by 163
- 2020Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuningcited by 153
- 2020Robust Pre-Training by Adversarial Contrastive Learningcited by 144
- 2020Adversarial Self-Supervised Contrastive Learningcited by 138
- 2020MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radiuscited by 114
- 2021Can contrastive learning avoid shortcut solutions?cited by 114
- 2021Salient ImageNet: How to discover spurious features in Deep Learning?cited by 114
- 2020Image Representations Learned With Unsupervised Pre-Training Contain Human-like Biasescited by 107
- 2020Contrastive Learning with Adversarial Examplescited by 91
- 2021When Does Contrastive Learning Preserve Adversarial Robustness from Pretraining to Finetuning?cited by 76
- 2019Wasserstein Dependency Measure for Representation Learningcited by 69
50 most cited of 56 All 56 (JSON)
Built on by
Citing papers per year: 2024: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2021Robust Learning Meets Generative Models: Can Proxy Distributions Improve Adversarial Robustness?
- 2025Evaluating Adversarial Robustness of Concept Representations in Sparse Autoencoders
- 2022Towards Alternative Techniques for Improving Adversarial Robustness: Analysis of Adversarial Training at a Spectrum of Perturbations
- 2023Adversarial Examples Are Not Real Features
- 2023Raising the Bar for Certified Adversarial Robustness with Diffusion Models
- 2025Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyond
- 2025RUB: Evaluating Residual Knowledge in Unlearned Models
- 2024Ensemble everything everywhere: Multi-scale aggregation for adversarial robustness