# PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples

arXiv: https://arxiv.org/abs/1710.10766 · HTML: https://mlsanity.com/paper/1710.10766/

## Key facts

- Cited by 244 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Oct 2017.

## Built on by (most cited first, 30 of 244)

- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [Maximum Likelihood Training of Score-Based Diffusion Models](https://mlsanity.com/paper/2101.09258/) (2021)
- [Diffusion Models for Adversarial Purification](https://mlsanity.com/paper/2205.07460/) (2022)
- [Better Diffusion Models Further Improve Adversarial Training](https://mlsanity.com/paper/2302.04638/) (2023)
- [Adversarial purification with Score-based generative models](https://mlsanity.com/paper/2106.06041/) (2021)
- [Guided Diffusion Model for Adversarial Purification](https://mlsanity.com/paper/2205.14969/) (2022)
- [GMMSeg: Gaussian Mixture based Generative Semantic Segmentation Models](https://mlsanity.com/paper/2210.02025/) (2022)
- [Self-Supervised Video Forensics by Audio-Visual Anomaly Detection](https://mlsanity.com/paper/2301.01767/) (2023)
- [Robust Classification via a Single Diffusion Model](https://mlsanity.com/paper/2305.15241/) (2023)
- [Robust Evaluation of Diffusion-Based Adversarial Purification](https://mlsanity.com/paper/2303.09051/) (2023)
- [Evaluating the Adversarial Robustness of Adaptive Test-time Defenses](https://mlsanity.com/paper/2202.13711/) (2022)
- [Adversarial Example Detection for DNN Models: A Review and Experimental Comparison](https://mlsanity.com/paper/2105.00203/) (2021)
- [Diffusion Models With Learned Adaptive Noise](https://mlsanity.com/paper/2312.13236/) (2023)
- [Online Adversarial Purification based on Self-Supervision](https://mlsanity.com/paper/2101.09387/) (2021)
- [Reveal of Vision Transformers Robustness against Adversarial Attacks](https://mlsanity.com/paper/2106.03734/) (2021)
- [Subspace Adversarial Training](https://mlsanity.com/paper/2111.12229/) (2021)
- [Boosting Fast Adversarial Training with Learnable Adversarial Initialization](https://mlsanity.com/paper/2110.05007/) (2021)
- [Biometrics: Trust, but Verify](https://mlsanity.com/paper/2105.06625/) (2021)
- [DiffAttack: Evasion Attacks Against Diffusion-Based Adversarial Purification](https://mlsanity.com/paper/2311.16124/) (2023)
- [Guided Diffusion Model for Adversarial Purification from Random Noise](https://mlsanity.com/paper/2206.10875/) (2022)
- [Improving Fast Adversarial Training with Prior-Guided Knowledge](https://mlsanity.com/paper/2304.00202/) (2023)
- [DISCO: Adversarial Defense with Local Implicit Functions](https://mlsanity.com/paper/2212.05630/) (2022)
- [DensePure: Understanding Diffusion Models towards Adversarial Robustness](https://mlsanity.com/paper/2211.00322/) (2022)
- [Why Does Little Robustness Help? A Further Step Towards Understanding Adversarial Transferability](https://mlsanity.com/paper/2307.07873/) (2023)
- [PatchZero: Defending against Adversarial Patch Attacks by Detecting and Zeroing the Patch](https://mlsanity.com/paper/2207.01795/) (2022)
- [Towards Efficient Adversarial Training on Vision Transformers](https://mlsanity.com/paper/2207.10498/) (2022)
- [Low Curvature Activations Reduce Overfitting in Adversarial Training](https://mlsanity.com/paper/2102.07861/) (2021)
- [Detecting Adversarial Examples from Sensitivity Inconsistency of Spatial-Transform Domain](https://mlsanity.com/paper/2103.04302/) (2021)
- [Drawing Robust Scratch Tickets: Subnetworks with Inborn Robustness Are Found within Randomly Initialized Networks](https://mlsanity.com/paper/2110.14068/) (2021)
- [A Review of Adversarial Attack and Defense for Classification Methods](https://mlsanity.com/paper/2111.09961/) (2021)

## Abstract (arXiv)

Adversarial perturbations of normal images are usually imperceptible to humans, but they can seriously confuse state-of-the-art machine learning models. What makes them so special in the eyes of image classifiers? In this paper, we show empirically that adversarial examples mainly lie in the low probability regions of the training distribution, regardless of attack types and targeted models. Using statistical hypothesis testing, we find that modern neural density models are surprisingly good at detecting imperceptible image perturbations. Based on this discovery, we devised PixelDefend, a new approach that purifies a maliciously perturbed image by moving it back towards the distribution seen in the training data. The purified image is then run through an unmodified classifier, making our method agnostic to both the classifier and the attacking method. As a result, PixelDefend can be used to protect already deployed models and be combined with other model-specific defenses. Experiments show that our method greatly improves resilience across a wide variety of state-of-the-art attacking methods, increasing accuracy on the strongest attack from 63% to 84% for Fashion MNIST and from 32% to 70% for CIFAR-10.