# Sample Efficient Detection and Classification of Adversarial Attacks via Self-Supervised Embeddings

arXiv: https://arxiv.org/abs/2108.13797 · HTML: https://mlsanity.com/paper/2108.13797/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 40 references that are in the map.
- First posted on arXiv 30 Aug 2021.

## Builds on (most cited first, 29 of 29)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [Certified Adversarial Robustness via Randomized Smoothing](https://mlsanity.com/paper/1902.02918/) (2019)
- [Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples](https://mlsanity.com/paper/1802.00420/) (2018)
- [Ensemble Adversarial Training: Attacks and Defenses](https://mlsanity.com/paper/1705.07204/) (2017)
- [Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods](https://mlsanity.com/paper/1705.07263/) (2017)
- [Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks](https://mlsanity.com/paper/1704.01155/) (2017)
- [Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality](https://mlsanity.com/paper/1801.02613/) (2018)
- [SSD: A Unified Framework for Self-Supervised Outlier Detection](https://mlsanity.com/paper/2103.12051/) (2021)
- [PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples](https://mlsanity.com/paper/1710.10766/) (2017)
- [On Detecting Adversarial Perturbations](https://mlsanity.com/paper/1702.04267/) (2017)
- [Spatially Transformed Adversarial Examples](https://mlsanity.com/paper/1801.02612/) (2018)
- [On the (Statistical) Detection of Adversarial Examples](https://mlsanity.com/paper/1702.06280/) (2017)
- [Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses](https://mlsanity.com/paper/2012.10544/) (2020)
- [Robust Pre-Training by Adversarial Contrastive Learning](https://mlsanity.com/paper/2010.13337/) (2020)
- [Adversarial Self-Supervised Contrastive Learning](https://mlsanity.com/paper/2006.07589/) (2020)
- [Perceptual Adversarial Robustness: Defense Against Unseen Threat Models](https://mlsanity.com/paper/2006.12655/) (2020)
- [Transferable Clean-Label Poisoning Attacks on Deep Neural Nets](https://mlsanity.com/paper/1905.05897/) (2019)
- [Just How Toxic is Data Poisoning? A Unified Benchmark for Backdoor and Data Poisoning Attacks](https://mlsanity.com/paper/2006.12557/) (2020)
- [Contrastive Learning with Adversarial Examples](https://mlsanity.com/paper/2010.12050/) (2020)
- [Functional Adversarial Attacks](https://mlsanity.com/paper/1906.00001/) (2019)
- [The Odds are Odd: A Statistical Test for Detecting Adversarial Examples](https://mlsanity.com/paper/1902.04818/) (2019)
- [Bullseye Polytope: A Scalable Clean-Label Poisoning Attack with Improved Transferability](https://mlsanity.com/paper/2005.00191/) (2020)
- [A Framework For Contrastive Self-Supervised Learning And Designing A New Approach](https://mlsanity.com/paper/2009.00104/) (2020)

## Built on by (most cited first, 11 of 11)

- [PRIME: A few primitives can boost robustness to common corruptions](https://mlsanity.com/paper/2112.13547/) (2021)
- [Explicit Tradeoffs between Adversarial and Natural Distributional Robustness](https://mlsanity.com/paper/2209.07592/) (2022)
- [Defenses in Adversarial Machine Learning: A Survey](https://mlsanity.com/paper/2312.08890/) (2023)
- [Perturbation Inactivation Based Adversarial Defense for Face Recognition](https://mlsanity.com/paper/2207.06035/) (2022)
- [Identifying Adversarial Attacks on Text Classifiers](https://mlsanity.com/paper/2201.08555/) (2022)
- [Self-Supervised Representation Learning for Adversarial Attack Detection](https://mlsanity.com/paper/2407.04382/) (2024)
- [Scalable Attribution of Adversarial Attacks via Multi-Task Learning](https://mlsanity.com/paper/2302.14059/) (2023)
- [Artificial Immune System of Secure Face Recognition Against Adversarial Attacks](https://mlsanity.com/paper/2406.18144/) (2024)
- [An Ambiguity Measure for Recognizing the Unknowns in Deep Learning](https://mlsanity.com/paper/2312.06077/) (2023)
- [A Linearly Convergent GAN Inversion-based Algorithm for Reverse Engineering of Deceptions](https://mlsanity.com/paper/2306.04756/) (2023)
- [SEA: Shareable and Explainable Attribution for Query-based Black-box Attacks](https://mlsanity.com/paper/2308.11845/) (2023)

## Abstract (arXiv)

Adversarial robustness of deep models is pivotal in ensuring safe deployment in real world settings, but most modern defenses have narrow scope and expensive costs. In this paper, we propose a self-supervised method to detect adversarial attacks and classify them to their respective threat models, based on a linear model operating on the embeddings from a pre-trained self-supervised encoder. We use a SimCLR encoder in our experiments, since we show the SimCLR embedding distance is a good proxy for human perceptibility, enabling it to encapsulate many threat models at once. We call our method SimCat since it uses SimCLR encoder to catch and categorize various types of adversarial attacks, including L_p and non-L_p evasion attacks, as well as data poisonings. The simple nature of a linear classifier makes our method efficient in both time and sample complexity. For example, on SVHN, using only five pairs of clean and adversarial examples computed with a PGD-L_inf attack, SimCat's detection accuracy is over 85%. Moreover, on ImageNet, using only 25 examples from each threat model, SimCat can classify eight different attack types such as PGD-L_2, PGD-L_inf, CW-L_2, PPGD, LPA, StAdv, ReColor, and JPEG-L_inf, with over 40% accuracy. On STL10 data, we apply SimCat as a defense against poisoning attacks, such as BP, CP, FC, CLBD, HTBD, halving the success rate while using only twenty total poisons for training. We find that the detectors generalize well to unseen threat models. Lastly, we investigate the performance of our detection method under adaptive attacks and further boost its robustness against such attacks via adversarial training.