# Adversarial Attacks are Reversible with Natural Supervision

arXiv: https://arxiv.org/abs/2103.14222 · HTML: https://mlsanity.com/paper/2103.14222/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Builds on 53 of its 71 references that are in the map.
- First posted on arXiv 26 Mar 2021.

## Builds on (most cited first, 30 of 53)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks](https://mlsanity.com/paper/2003.01690/) (2020)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [DeepFool: a simple and accurate method to fool deep neural networks](https://mlsanity.com/paper/1511.04599/) (2015)
- [Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples](https://mlsanity.com/paper/1802.00420/) (2018)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [Robustness May Be at Odds with Accuracy](https://mlsanity.com/paper/1805.12152/) (2018)
- [Self-Supervised Learning of Pretext-Invariant Representations](https://mlsanity.com/paper/1912.01991/) (2019)
- [The Limitations of Deep Learning in Adversarial Settings](https://mlsanity.com/paper/1511.07528/) (2015)
- [Fast is better than free: Revisiting adversarial training](https://mlsanity.com/paper/2001.03994/) (2020)

## Built on by (most cited first, 30 of 47)

- [MEMO: Test Time Robustness via Adaptation and Augmentation](https://mlsanity.com/paper/2110.09506/) (2021)
- [A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts](https://mlsanity.com/paper/2303.15361/) (2023)
- [Understanding Zero-Shot Adversarial Robustness for Large-Scale Models](https://mlsanity.com/paper/2212.07016/) (2022)
- [Visual-RolePlay: Universal Jailbreak Attack on MultiModal Large Language Models via Role-playing Image Character](https://mlsanity.com/paper/2405.20773/) (2024)
- [How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defenses](https://mlsanity.com/paper/2305.10862/) (2023)
- [Evaluating the Adversarial Robustness of Adaptive Test-time Defenses](https://mlsanity.com/paper/2202.13711/) (2022)
- [Visual Prompting for Adversarial Robustness](https://mlsanity.com/paper/2210.06284/) (2022)
- [DISCO: Adversarial Defense with Local Implicit Functions](https://mlsanity.com/paper/2212.05630/) (2022)
- [Self-Interpretable Time Series Prediction with Counterfactual Explanations](https://mlsanity.com/paper/2306.06024/) (2023)
- [CLIP is Strong Enough to Fight Back: Test-time Counterattacks towards Zero-shot Adversarial Robustness of CLIP](https://mlsanity.com/paper/2503.03613/) (2025)
- [Convolutional Visual Prompt for Robust Visual Perception](https://mlsanity.com/paper/2303.00198/) (2023)
- [Landscape Learning for Neural Network Inversion](https://mlsanity.com/paper/2206.09027/) (2022)
- [AutoAdvExBench: Benchmarking autonomous exploitation of adversarial example defenses](https://mlsanity.com/paper/2503.01811/) (2025)
- [Enhancing CLIP Robustness via Cross-Modality Alignment](https://mlsanity.com/paper/2510.24038/) (2025)
- [Defenses in Adversarial Machine Learning: A Survey](https://mlsanity.com/paper/2312.08890/) (2023)
- [GDA: Generalized Diffusion for Robust Test-time Adaptation](https://mlsanity.com/paper/2404.00095/) (2024)
- [Robust Perception through Equivariance](https://mlsanity.com/paper/2212.06079/) (2022)
- [Adaptive Randomized Smoothing: Certified Adversarial Robustness for Multi-Step Defences](https://mlsanity.com/paper/2406.10427/) (2024)
- [Adversarial Purification with the Manifold Hypothesis](https://mlsanity.com/paper/2210.14404/) (2022)
- [Real-Time Neural Voice Camouflage](https://mlsanity.com/paper/2112.07076/) (2021)
- [Taxonomy-Structured Domain Adaptation](https://mlsanity.com/paper/2306.07874/) (2023)
- [PuriDefense: Randomized Local Implicit Adversarial Purification for Defending Black-box Query-based Attacks](https://mlsanity.com/paper/2401.10586/) (2024)
- [Be Your Own Neighborhood: Detecting Adversarial Example by the Neighborhood Relations Built on Self-Supervised Learning](https://mlsanity.com/paper/2209.00005/) (2022)
- [Meta Invariance Defense Towards Generalizable Robustness to Unknown Adversarial Attacks](https://mlsanity.com/paper/2404.03340/) (2024)
- [Using Multiple Self-Supervised Tasks Improves Model Robustness](https://mlsanity.com/paper/2204.03714/) (2022)
- [Test-time Detection and Repair of Adversarial Samples via Masked Autoencoder](https://mlsanity.com/paper/2303.12848/) (2023)
- [Robust Feature Inference: A Test-time Defense Strategy using Spectral Projections](https://mlsanity.com/paper/2307.11672/) (2023)
- [Interpretability-Guided Test-Time Adversarial Defense](https://mlsanity.com/paper/2409.15190/) (2024)
- [Adversarially Robust Video Perception by Seeing Motion](https://mlsanity.com/paper/2212.07815/) (2022)
- [Improving Adversarial Robustness of DEQs with Explicit Regulations Along the Neural Dynamics](https://mlsanity.com/paper/2306.01435/) (2023)

## Abstract (arXiv)

We find that images contain intrinsic structure that enables the reversal of many adversarial attacks. Attack vectors cause not only image classifiers to fail, but also collaterally disrupt incidental structure in the image. We demonstrate that modifying the attacked image to restore the natural structure will reverse many types of attacks, providing a defense. Experiments demonstrate significantly improved robustness for several state-of-the-art models across the CIFAR-10, CIFAR-100, SVHN, and ImageNet datasets. Our results show that our defense is still effective even if the attacker is aware of the defense mechanism. Since our defense is deployed during inference instead of training, it is compatible with pre-trained networks as well as most other defenses. Our results suggest deep networks are vulnerable to adversarial examples partly because their representations do not enforce the natural structure of images.