# Fooling Adversarial Training with Inducing Noise

arXiv: https://arxiv.org/abs/2111.10130 · HTML: https://mlsanity.com/paper/2111.10130/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 26 references that are in the map.
- First posted on arXiv 19 Nov 2021.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples](https://mlsanity.com/paper/1802.00420/) (2018)
- [Adversarial Examples Are Not Bugs, They Are Features](https://mlsanity.com/paper/1905.02175/) (2019)
- [Adversarial Machine Learning at Scale](https://mlsanity.com/paper/1611.01236/) (2016)
- [Adversarial Attacks and Defences: A Survey](https://mlsanity.com/paper/1810.00069/) (2018)
- [Towards Poisoning of Deep Learning Algorithms with Back-gradient Optimization](https://mlsanity.com/paper/1708.08689/) (2017)
- [Unlearnable Examples: Making Personal Data Unexploitable](https://mlsanity.com/paper/2101.04898/) (2021)
- [Understanding Adversarial Attacks on Deep Learning Based Medical Image Analysis Systems](https://mlsanity.com/paper/1907.10456/) (2019)
- [Adversarial Examples Make Strong Poisons](https://mlsanity.com/paper/2106.10807/) (2021)
- [Learning to Confuse: Generating Training Time Adversarial Data with Auto-Encoder](https://mlsanity.com/paper/1905.09027/) (2019)
- [Better Safe Than Sorry: Preventing Delusive Adversaries with Adversarial Training](https://mlsanity.com/paper/2102.04716/) (2021)
- [Bilateral Adversarial Training: Towards Fast Training of More Robust Models Against Adversarial Attacks](https://mlsanity.com/paper/1811.10716/) (2018)
- [Preventing Unauthorized Use of Proprietary Data: Poisoning for Secure Dataset Release](https://mlsanity.com/paper/2103.02683/) (2021)
- [Analysis and Applications of Class-wise Robustness in Adversarial Training](https://mlsanity.com/paper/2105.14240/) (2021)
- [Disrupting Model Training with Adversarial Shortcuts](https://mlsanity.com/paper/2106.06654/) (2021)

## Built on by (most cited first, 13 of 13)

- [Autoregressive Perturbations for Data Poisoning](https://mlsanity.com/paper/2206.03693/) (2022)
- [Image Shortcut Squeezing: Countering Perturbative Availability Poisons with Compression](https://mlsanity.com/paper/2301.13838/) (2023)
- [Balance, Imbalance, and Rebalance: Understanding Robust Overfitting from a Minimax Game Perspective](https://mlsanity.com/paper/2310.19360/) (2023)
- [Unlearnable Examples Give a False Sense of Security: Piercing through Unexploitable Data with Learnable Examples](https://mlsanity.com/paper/2305.09241/) (2023)
- [Poisons that are learned faster are more effective](https://mlsanity.com/paper/2204.08615/) (2022)
- [Detection and Defense of Unlearnable Examples](https://mlsanity.com/paper/2312.08898/) (2023)
- [Semantic Deep Hiding for Robust Unlearnable Examples](https://mlsanity.com/paper/2406.17349/) (2024)
- [Can Adversarial Training Be Manipulated By Non-Robust Features?](https://mlsanity.com/paper/2201.13329/) (2022)
- [ECLIPSE: Expunging Clean-label Indiscriminate Poisons via Sparse Diffusion Purification](https://mlsanity.com/paper/2406.15093/) (2024)
- [Nonlinear Transformations Against Unlearnable Datasets](https://mlsanity.com/paper/2406.02883/) (2024)
- [Data-Dependent Stability Analysis of Adversarial Training](https://mlsanity.com/paper/2401.03156/) (2024)
- [Channel-Level Semantic Perturbations: Unlearnable Examples for Diverse Training Paradigms](https://mlsanity.com/paper/2605.05224/) (2026)
- [SoK: A Comprehensive Analysis of the Current Status of Neural Tangent Generalization Attacks with Research Directions](https://mlsanity.com/paper/2605.12792/) (2026)

## Abstract (arXiv)

Adversarial training is widely believed to be a reliable approach to improve model robustness against adversarial attack. However, in this paper, we show that when trained on one type of poisoned data, adversarial training can also be fooled to have catastrophic behavior, e.g., <1% robust test accuracy with >90% robust training accuracy on CIFAR-10 dataset. Previously, there are other types of noise poisoned in the training data that have successfully fooled standard training (15.8% standard test accuracy with 99.9% standard training accuracy on CIFAR-10 dataset), but their poisonings can be easily removed when adopting adversarial training. Therefore, we aim to design a new type of inducing noise, named ADVIN, which is an irremovable poisoning of training data. ADVIN can not only degrade the robustness of adversarial training by a large margin, for example, from 51.7% to 0.57% on CIFAR-10 dataset, but also be effective for fooling standard training (13.1% standard test accuracy with 100% standard training accuracy). Additionally, ADVIN can be applied to preventing personal data (like selfies) from being exploited without authorization under whether standard or adversarial training.