# Learning to Confuse: Generating Training Time Adversarial Data with Auto-Encoder

arXiv: https://arxiv.org/abs/1905.09027 · HTML: https://mlsanity.com/paper/1905.09027/

## Key facts

- Cited by 70 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 May 2019.

## Built on by (most cited first, 30 of 70)

- [Adversarial Examples Make Strong Poisons](https://mlsanity.com/paper/2106.10807/) (2021)
- [Sleeper Agent: Scalable Hidden Trigger Backdoors for Neural Networks Trained from Scratch](https://mlsanity.com/paper/2106.08970/) (2021)
- [Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning](https://mlsanity.com/paper/2205.01992/) (2022)
- [Untargeted Backdoor Watermark: Towards Harmless and Stealthy Dataset Copyright Protection](https://mlsanity.com/paper/2210.00875/) (2022)
- [Availability Attacks Create Shortcuts](https://mlsanity.com/paper/2111.00898/) (2021)
- [Better Safe Than Sorry: Preventing Delusive Adversaries with Adversarial Training](https://mlsanity.com/paper/2102.04716/) (2021)
- [MetaCloak: Preventing Unauthorized Subject-driven Text-to-image Diffusion-based Synthesis via Meta-learning](https://mlsanity.com/paper/2311.13127/) (2023)
- [Image Shortcut Squeezing: Countering Perturbative Availability Poisons with Compression](https://mlsanity.com/paper/2301.13838/) (2023)
- [Initiative Defense against Facial Manipulation](https://mlsanity.com/paper/2112.10098/) (2021)
- [Unlearnable Clusters: Towards Label-agnostic Unlearnable Examples](https://mlsanity.com/paper/2301.01217/) (2022)
- [Indiscriminate Poisoning Attacks on Unsupervised Contrastive Learning](https://mlsanity.com/paper/2202.11202/) (2022)
- [Energy-Latency Attacks via Sponge Poisoning](https://mlsanity.com/paper/2203.08147/) (2022)
- [One-Pixel Shortcut: on the Learning Preference of Deep Neural Networks](https://mlsanity.com/paper/2205.12141/) (2022)
- [One-shot Neural Backdoor Erasing via Adversarial Weight Masking](https://mlsanity.com/paper/2207.04497/) (2022)
- [Stable Unlearnable Example: Enhancing the Robustness of Unlearnable Examples via Stable Error-Minimizing Noise](https://mlsanity.com/paper/2311.13091/) (2023)
- [Machine Learning Security against Data Poisoning: Are We There Yet?](https://mlsanity.com/paper/2204.05986/) (2022)
- [What Can We Learn from Unlearnable Datasets?](https://mlsanity.com/paper/2305.19254/) (2023)
- [Data Poisoning in Deep Learning: A Survey](https://mlsanity.com/paper/2503.22759/) (2025)
- [Self-Ensemble Protection: Training Checkpoints Are Good Data Protectors](https://mlsanity.com/paper/2211.12005/) (2022)
- [The Devil's Advocate: Shattering the Illusion of Unexploitable Data using Diffusion Models](https://mlsanity.com/paper/2303.08500/) (2023)
- [Purify Unlearnable Examples via Rate-Constrained Variational Autoencoders](https://mlsanity.com/paper/2405.01460/) (2024)
- [Provably Unlearnable Data Examples](https://mlsanity.com/paper/2405.03316/) (2024)
- [Unlearnable Examples Give a False Sense of Security: Piercing through Unexploitable Data with Learnable Examples](https://mlsanity.com/paper/2305.09241/) (2023)
- [Adversarial Reinforced Instruction Attacker for Robust Vision-Language Navigation](https://mlsanity.com/paper/2107.11252/) (2021)
- [Trustworthy Distributed AI Systems: Robustness, Privacy, and Governance](https://mlsanity.com/paper/2402.01096/) (2024)
- [Fooling Adversarial Training with Inducing Noise](https://mlsanity.com/paper/2111.10130/) (2021)
- [Detection and Defense of Unlearnable Examples](https://mlsanity.com/paper/2312.08898/) (2023)
- [Semantic Deep Hiding for Robust Unlearnable Examples](https://mlsanity.com/paper/2406.17349/) (2024)
- [Can Adversarial Training Be Manipulated By Non-Robust Features?](https://mlsanity.com/paper/2201.13329/) (2022)
- [A Security Risk Taxonomy for Prompt-Based Interaction With Large Language Models](https://mlsanity.com/paper/2311.11415/) (2023)

## Abstract (arXiv)

In this work, we consider one challenging training time attack by modifying training data with bounded perturbation, hoping to manipulate the behavior (both targeted or non-targeted) of any corresponding trained classifier during test time when facing clean samples. To achieve this, we proposed to use an auto-encoder-like network to generate the pertubation on the training data paired with one differentiable system acting as the imaginary victim classifier. The perturbation generator will learn to update its weights by watching the training procedure of the imaginary classifier in order to produce the most harmful and imperceivable noise which in turn will lead the lowest generalization power for the victim classifier. This can be formulated into a non-linear equality constrained optimization problem. Unlike GANs, solving such problem is computationally challenging, we then proposed a simple yet effective procedure to decouple the alternating updates for the two networks for stability. The method proposed in this paper can be easily extended to the label specific setting where the attacker can manipulate the predictions of the victim classifiers according to some predefined rules rather than only making wrong predictions. Experiments on various datasets including CIFAR-10 and a reduced version of ImageNet confirmed the effectiveness of the proposed method and empirical results showed that, such bounded perturbation have good transferability regardless of which classifier the victim is actually using on image data.