# Learning Defense Transformers for Counterattacking Adversarial Examples

arXiv: https://arxiv.org/abs/2103.07595 · HTML: https://mlsanity.com/paper/2103.07595/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 41 references that are in the map.
- First posted on arXiv 13 Mar 2021.

## Builds on (most cited first, 30 of 33)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [DeepFool: a simple and accurate method to fool deep neural networks](https://mlsanity.com/paper/1511.04599/) (2015)
- [One pixel attack for fooling deep neural networks](https://mlsanity.com/paper/1710.08864/) (2017)
- [Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement](https://mlsanity.com/paper/2001.06826/) (2020)
- [Countering Adversarial Images using Input Transformations](https://mlsanity.com/paper/1711.00117/) (2017)
- [Defense against Adversarial Attacks Using High-Level Representation Guided Denoiser](https://mlsanity.com/paper/1712.02976/) (2017)
- [Feature Denoising for Improving Adversarial Robustness](https://mlsanity.com/paper/1812.03411/) (2018)
- [Adversarial Logit Pairing](https://mlsanity.com/paper/1803.06373/) (2018)
- [Spatially Transformed Adversarial Examples](https://mlsanity.com/paper/1801.02612/) (2018)
- [Adversarial Training and Robustness for Multiple Perturbations](https://mlsanity.com/paper/1904.13000/) (2019)
- [You Only Propagate Once: Accelerating Adversarial Training via Maximal Principle](https://mlsanity.com/paper/1905.00877/) (2019)
- [Scaling provable adversarial defenses](https://mlsanity.com/paper/1805.12514/) (2018)
- [DeepLPF: Deep Local Parametric Filters for Image Enhancement](https://mlsanity.com/paper/2003.13985/) (2020)
- [Feature Distillation: DNN-Oriented JPEG Compression Against Adversarial Examples](https://mlsanity.com/paper/1803.05787/) (2018)
- [Functional Adversarial Attacks](https://mlsanity.com/paper/1906.00001/) (2019)
- [Deep Relational Reasoning Graph Network for Arbitrary Shape Text Detection](https://mlsanity.com/paper/2003.07493/) (2020)
- [ContourNet: Taking a Further Step toward Accurate Arbitrary-shaped Scene Text Detection](https://mlsanity.com/paper/2004.04940/) (2020)
- [DaST: Data-free Substitute Training for Adversarial Attacks](https://mlsanity.com/paper/2003.12703/) (2020)
- [Discriminative Multi-modality Speech Recognition](https://mlsanity.com/paper/2005.05592/) (2020)
- [Mixup Inference: Better Exploiting Mixup to Defend Adversarial Attacks](https://mlsanity.com/paper/1909.11515/) (2019)
- [On the Robustness of the CVPR 2018 White-Box Adversarial Example Defenses](https://mlsanity.com/paper/1804.03286/) (2018)
- [ADef: an Iterative Algorithm to Construct Adversarial Deformations](https://mlsanity.com/paper/1804.07729/) (2018)

## Built on by (most cited first, 2 of 2)

- [Inference Time Evidences of Adversarial Attacks for Forensic on Transformers](https://mlsanity.com/paper/2301.13356/) (2023)
- [Parameter-Saving Adversarial Training: Reinforcing Multi-Perturbation Robustness via Hypernetworks](https://mlsanity.com/paper/2309.16207/) (2023)

## Abstract (arXiv)

Deep neural networks (DNNs) are vulnerable to adversarial examples with small perturbations. Adversarial defense thus has been an important means which improves the robustness of DNNs by defending against adversarial examples. Existing defense methods focus on some specific types of adversarial examples and may fail to defend well in real-world applications. In practice, we may face many types of attacks where the exact type of adversarial examples in real-world applications can be even unknown. In this paper, motivated by that adversarial examples are more likely to appear near the classification boundary, we study adversarial examples from a new perspective that whether we can defend against adversarial examples by pulling them back to the original clean distribution. We theoretically and empirically verify the existence of defense affine transformations that restore adversarial examples. Relying on this, we learn a defense transformer to counterattack the adversarial examples by parameterizing the affine transformations and exploiting the boundary information of DNNs. Extensive experiments on both toy and real-world datasets demonstrate the effectiveness and generalization of our defense transformer.