# What Doesn't Kill You Makes You Robust(er): How to Adversarially Train against Data Poisoning

arXiv: https://arxiv.org/abs/2102.13624 · HTML: https://mlsanity.com/paper/2102.13624/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security.
- Builds on 39 of its 50 references that are in the map.
- First posted on arXiv 26 Feb 2021.

## Placed in

- LLM safety and security / [Backdoor and data poisoning attacks](https://mlsanity.com/safety/backdoor-and-data-poisoning-attacks/)

## Builds on (most cited first, 30 of 39)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks](https://mlsanity.com/paper/1805.12185/) (2018)
- [On Adaptive Attacks to Adversarial Example Defenses](https://mlsanity.com/paper/2002.08347/) (2020)
- [Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networks](https://mlsanity.com/paper/1804.00792/) (2018)
- [Spectral Signatures in Backdoor Attacks](https://mlsanity.com/paper/1811.00636/) (2018)
- [STRIP: A Defence Against Trojan Attacks on Deep Neural Networks](https://mlsanity.com/paper/1902.06531/) (2019)
- [Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering](https://mlsanity.com/paper/1811.03728/) (2018)
- [On Evaluating Adversarial Robustness](https://mlsanity.com/paper/1902.06705/) (2019)
- [Hidden Trigger Backdoor Attacks](https://mlsanity.com/paper/1910.00033/) (2019)
- [Certifying Some Distributional Robustness with Principled Adversarial Training](https://mlsanity.com/paper/1710.10571/) (2017)
- [Detecting AI Trojans Using Meta Neural Analysis](https://mlsanity.com/paper/1910.03137/) (2019)
- [Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses](https://mlsanity.com/paper/2012.10544/) (2020)
- [Witches' Brew: Industrial Scale Data Poisoning via Gradient Matching](https://mlsanity.com/paper/2009.02276/) (2020)
- [Stronger Data Poisoning Attacks Break Data Sanitization Defenses](https://mlsanity.com/paper/1811.00741/) (2018)
- [Transferable Clean-Label Poisoning Attacks on Deep Neural Nets](https://mlsanity.com/paper/1905.05897/) (2019)
- [Adversarial Machine Learning -- Industry Perspectives](https://mlsanity.com/paper/2002.05646/) (2020)
- [MetaPoison: Practical General-purpose Clean-label Data Poisoning](https://mlsanity.com/paper/2004.00225/) (2020)
- [RAB: Provable Robustness Against Backdoor Attacks](https://mlsanity.com/paper/2003.08904/) (2020)
- [Just How Toxic is Data Poisoning? A Unified Benchmark for Backdoor and Data Poisoning Attacks](https://mlsanity.com/paper/2006.12557/) (2020)
- [Deep Partition Aggregation: Provable Defense against General Poisoning Attacks](https://mlsanity.com/paper/2006.14768/) (2020)
- [Strong Data Augmentation Sanitizes Poisoning and Backdoor Attacks Without an Accuracy Tradeoff](https://mlsanity.com/paper/2011.09527/) (2020)
- [On the Effectiveness of Mitigating Data Poisoning Attacks with Gradient Shaping](https://mlsanity.com/paper/2002.11497/) (2020)

## Built on by (most cited first, 22 of 22)

- [Poison Ink: Robust and Invisible Backdoor Attack](https://mlsanity.com/paper/2108.02488/) (2021)
- [Shadowcast: Stealthy Data Poisoning Attacks Against Vision-Language Models](https://mlsanity.com/paper/2402.06659/) (2024)
- [Immunization against harmful fine-tuning attacks](https://mlsanity.com/paper/2402.16382/) (2024)
- [Better Safe than Sorry: Pre-training CLIP against Targeted Data Poisoning and Backdoor Attacks](https://mlsanity.com/paper/2310.05862/) (2023)
- [On the Effectiveness of Adversarial Training against Backdoor Attacks](https://mlsanity.com/paper/2202.10627/) (2022)
- [Towards Unified Robustness Against Both Backdoor and Adversarial Attacks](https://mlsanity.com/paper/2405.17929/) (2024)
- [Adversarial Training for Defense Against Label Poisoning Attacks](https://mlsanity.com/paper/2502.17121/) (2025)
- [Backdoor Collapse: Eliminating Unknown Threats via Known Backdoor Aggregation in Language Models](https://mlsanity.com/paper/2510.10265/) (2025)
- [SEEP: Training Dynamics Grounds Latent Representation Search for Mitigating Backdoor Poisoning Attacks](https://mlsanity.com/paper/2405.11575/) (2024)
- [Neuroplasticity and Corruption in Model Mechanisms: A Case Study Of Indirect Object Identification](https://mlsanity.com/paper/2503.01896/) (2025)
- [Towards Unveiling Predictive Uncertainty Vulnerabilities in the Context of the Right to Be Forgotten](https://mlsanity.com/paper/2508.07458/) (2025)
- [Clean Label Attacks against SLU Systems](https://mlsanity.com/paper/2409.08985/) (2024)
- [Mithridates: Auditing and Boosting Backdoor Resistance of Machine Learning Pipelines](https://mlsanity.com/paper/2302.04977/) (2023)
- [Pick your Poison: Undetectability versus Robustness in Data Poisoning Attacks](https://mlsanity.com/paper/2305.09671/) (2023)
- [HINT: Healthy Influential-Noise based Training to Defend against Data Poisoning Attacks](https://mlsanity.com/paper/2309.08549/) (2023)
- [Transparency Attacks: How Imperceptible Image Layers Can Fool AI Perception](https://mlsanity.com/paper/2401.15817/) (2024)
- [Certified Robustness to Clean-Label Poisoning Using Diffusion Denoising](https://mlsanity.com/paper/2403.11981/) (2024)
- [Have You Poisoned My Data? Defending Neural Networks against Data Poisoning](https://mlsanity.com/paper/2403.13523/) (2024)
- [Safety-Efficacy Trade Off: Robustness against Data-Poisoning](https://mlsanity.com/paper/2602.00822/) (2026)
- [Projected Subnetworks Scale Adaptation](https://mlsanity.com/paper/2301.11487/) (2023)
- [Exploiting Alpha Transparency In Language And Vision-Based AI Systems](https://mlsanity.com/paper/2402.09671/) (2024)
- [SoK: A Comprehensive Analysis of the Current Status of Neural Tangent Generalization Attacks with Research Directions](https://mlsanity.com/paper/2605.12792/) (2026)

## Abstract (arXiv)

Data poisoning is a threat model in which a malicious actor tampers with training data to manipulate outcomes at inference time. A variety of defenses against this threat model have been proposed, but each suffers from at least one of the following flaws: they are easily overcome by adaptive attacks, they severely reduce testing performance, or they cannot generalize to diverse data poisoning threat models. Adversarial training, and its variants, are currently considered the only empirically strong defense against (inference-time) adversarial attacks. In this work, we extend the adversarial training framework to defend against (training-time) data poisoning, including targeted and backdoor attacks. Our method desensitizes networks to the effects of such attacks by creating poisons during training and injecting them into training batches. We show that this defense withstands adaptive attacks, generalizes to diverse threat models, and incurs a better performance trade-off than previous defenses such as DP-SGD or (evasion) adversarial training.