# Just How Toxic is Data Poisoning? A Unified Benchmark for Backdoor and Data Poisoning Attacks

arXiv: https://arxiv.org/abs/2006.12557 · HTML: https://mlsanity.com/paper/2006.12557/

## Key facts

- Cited by 116 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jun 2020.

## Built on by (most cited first, 30 of 116)

- [Sleeper Agent: Scalable Hidden Trigger Backdoors for Neural Networks Trained from Scratch](https://mlsanity.com/paper/2106.08970/) (2021)
- [BadEdit: Backdooring large language models by model editing](https://mlsanity.com/paper/2403.13355/) (2024)
- [Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning](https://mlsanity.com/paper/2205.01992/) (2022)
- [Untargeted Backdoor Watermark: Towards Harmless and Stealthy Dataset Copyright Protection](https://mlsanity.com/paper/2210.00875/) (2022)
- [Nightshade: Prompt-Specific Poisoning Attacks on Text-to-Image Generative Models](https://mlsanity.com/paper/2310.13828/) (2023)
- [A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarks](https://mlsanity.com/paper/2206.08514/) (2022)
- [Availability Attacks Create Shortcuts](https://mlsanity.com/paper/2111.00898/) (2021)
- [Better Safe Than Sorry: Preventing Delusive Adversaries with Adversarial Training](https://mlsanity.com/paper/2102.04716/) (2021)
- [Hard to Forget: Poisoning Attacks on Certified Machine Unlearning](https://mlsanity.com/paper/2109.08266/) (2021)
- [Improved Certified Defenses against Data Poisoning with (Deterministic) Finite Aggregation](https://mlsanity.com/paper/2202.02628/) (2022)
- [Shadowcast: Stealthy Data Poisoning Attacks Against Vision-Language Models](https://mlsanity.com/paper/2402.06659/) (2024)
- [Commercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attacks](https://mlsanity.com/paper/2502.08586/) (2025)
- [Hidden Poison: Machine Unlearning Enables Camouflaged Poisoning Attacks](https://mlsanity.com/paper/2212.10717/) (2022)
- [Test-Time Backdoor Attacks on Multimodal Large Language Models](https://mlsanity.com/paper/2402.08577/) (2024)
- [Securing Large Language Models: Threats, Vulnerabilities and Responsible Practices](https://mlsanity.com/paper/2403.12503/) (2024)
- [Unlearnable Clusters: Towards Label-agnostic Unlearnable Examples](https://mlsanity.com/paper/2301.01217/) (2022)
- [DP-InstaHide: Provably Defusing Poisoning and Backdoor Attacks with Differentially Private Data Augmentations](https://mlsanity.com/paper/2103.02079/) (2021)
- [Defending Against Backdoor Attacks in Natural Language Generation](https://mlsanity.com/paper/2106.01810/) (2021)
- [Not All Poisons are Created Equal: Robust Training against Data Poisoning](https://mlsanity.com/paper/2210.09671/) (2022)
- [BadRL: Sparse Targeted Backdoor Attack Against Reinforcement Learning](https://mlsanity.com/paper/2312.12585/) (2023)
- [Label Poisoning is All You Need](https://mlsanity.com/paper/2310.18933/) (2023)
- [COPA: Certifying Robust Policies for Offline Reinforcement Learning against Poisoning Attacks](https://mlsanity.com/paper/2203.08398/) (2022)
- [Indiscriminate Data Poisoning Attacks on Neural Networks](https://mlsanity.com/paper/2204.09092/) (2022)
- [What Doesn't Kill You Makes You Robust(er): How to Adversarially Train against Data Poisoning](https://mlsanity.com/paper/2102.13624/) (2021)
- [Friendly Noise against Adversarial Noise: A Powerful Defense against Data Poisoning Attacks](https://mlsanity.com/paper/2208.10224/) (2022)
- [The Devil's Advocate: Shattering the Illusion of Unexploitable Data using Diffusion Models](https://mlsanity.com/paper/2303.08500/) (2023)
- [Data and Model Poisoning Backdoor Attacks on Wireless Federated Learning, and the Defense Mechanisms: A Comprehensive Survey](https://mlsanity.com/paper/2312.08667/) (2023)
- [Purify Unlearnable Examples via Rate-Constrained Variational Autoencoders](https://mlsanity.com/paper/2405.01460/) (2024)
- [An Equivalence Between Data Poisoning and Byzantine Gradient Attacks](https://mlsanity.com/paper/2202.08578/) (2022)
- [On Collective Robustness of Bagging Against Data Poisoning](https://mlsanity.com/paper/2205.13176/) (2022)

## Abstract (arXiv)

Data poisoning and backdoor attacks manipulate training data in order to cause models to fail during inference. A recent survey of industry practitioners found that data poisoning is the number one concern among threats ranging from model stealing to adversarial attacks. However, it remains unclear exactly how dangerous poisoning methods are and which ones are more effective considering that these methods, even ones with identical objectives, have not been tested in consistent or realistic settings. We observe that data poisoning and backdoor attacks are highly sensitive to variations in the testing setup. Moreover, we find that existing methods may not generalize to realistic settings. While these existing works serve as valuable prototypes for data poisoning, we apply rigorous tests to determine the extent to which we should fear them. In order to promote fair comparison in future work, we develop standardized benchmarks for data poisoning and backdoor attacks.