mlsanityData to 14 Oct 2026

Just How Toxic is Data Poisoning? A Unified Benchmark for Backdoor and Data Poisoning Attacks

arXiv 2006.12557 ↗First posted 22 Jun 2020cs.LG cs.CR cs.CVcited by 116

Not on the map. Why?

Cited by 49 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Data poisoning and backdoor attacks manipulate training data in order to cause models to fail during inference. A recent survey of industry practitioners found that data poisoning is the number one concern among threats ranging from model stealing to adversarial attacks. However, it remains unclear exactly how dangerous poisoning methods are and which ones are more effective considering that these methods, even ones with identical objectives, have not been tested in consistent or realistic settings. We observe that data poisoning and backdoor attacks are highly sensitive to variations in the testing setup. Moreover, we find that existing methods may not generalize to realistic settings. While these existing works serve as valuable prototypes for data poisoning, we apply rigorous tests to determine the extent to which we should fear them. In order to promote fair comparison in future work, we develop standardized benchmarks for data poisoning and backdoor attacks.

Key facts. Cited by 116 papers in the map (data through 14 Oct 2026). First posted on arXiv 22 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 16; 2022: 22; 2023: 18; 2024: 29; 2025: 16; 2026: 15.

116 papers.

Sort:

By area:

  1. 2021Sleeper Agent: Scalable Hidden Trigger Backdoors for Neural Networks Trained from Scratchcited by 102
  2. 2024BadEdit: Backdooring large language models by model editingcited by 96
  3. 2022Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoningcited by 94
  4. 2022Untargeted Backdoor Watermark: Towards Harmless and Stealthy Dataset Copyright Protectioncited by 86
  5. 2023Nightshade: Prompt-Specific Poisoning Attacks on Text-to-Image Generative Modelscited by 75
  6. 2022A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarkscited by 74
  7. 2021Availability Attacks Create Shortcutscited by 62
  8. 2021Better Safe Than Sorry: Preventing Delusive Adversaries with Adversarial Trainingcited by 59
  9. 2021Hard to Forget: Poisoning Attacks on Certified Machine Unlearningcited by 52
  10. 2022Improved Certified Defenses against Data Poisoning with (Deterministic) Finite Aggregationcited by 52
  11. 2024Shadowcast: Stealthy Data Poisoning Attacks Against Vision-Language Modelscited by 50
  12. 2025Commercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attackscited by 49
38 more
  1. 2022Hidden Poison: Machine Unlearning Enables Camouflaged Poisoning Attackscited by 43
  2. 2024Test-Time Backdoor Attacks on Multimodal Large Language Modelscited by 40
  3. 2024Securing Large Language Models: Threats, Vulnerabilities and Responsible Practicescited by 37
  4. 2022Unlearnable Clusters: Towards Label-agnostic Unlearnable Examplescited by 36
  5. 2021DP-InstaHide: Provably Defusing Poisoning and Backdoor Attacks with Differentially Private Data Augmentationscited by 34
  6. 2021Defending Against Backdoor Attacks in Natural Language Generationcited by 33
  7. 2022Not All Poisons are Created Equal: Robust Training against Data Poisoningcited by 30
  8. 2023BadRL: Sparse Targeted Backdoor Attack Against Reinforcement Learningcited by 30
  9. 2023Label Poisoning is All You Needcited by 29
  10. 2022COPA: Certifying Robust Policies for Offline Reinforcement Learning against Poisoning Attackscited by 26
  11. 2022Indiscriminate Data Poisoning Attacks on Neural Networkscited by 25
  12. 2021What Doesn't Kill You Makes You Robust(er): How to Adversarially Train against Data Poisoningcited by 22
  13. 2022Friendly Noise against Adversarial Noise: A Powerful Defense against Data Poisoning Attackscited by 21
  14. 2023The Devil's Advocate: Shattering the Illusion of Unexploitable Data using Diffusion Modelscited by 19
  15. 2023Data and Model Poisoning Backdoor Attacks on Wireless Federated Learning, and the Defense Mechanisms: A Comprehensive Surveycited by 18
  16. 2024Purify Unlearnable Examples via Rate-Constrained Variational Autoencoderscited by 18
  17. 2022An Equivalence Between Data Poisoning and Byzantine Gradient Attackscited by 16
  18. 2022On Collective Robustness of Bagging Against Data Poisoningcited by 16
  19. 2022Quarantine: Sparsity Can Uncover the Trojan Attack Trigger for Freecited by 15
  20. 2024Decoding the AI Pen: Techniques and Challenges in Detecting AI-Generated Textcited by 14
  21. 2023Detection and Defense of Unlearnable Examplescited by 13
  22. 2021Backdoor Attack on Hash-based Image Retrieval via Clean-label Data Poisoningcited by 12
  23. 2022Can Adversarial Training Be Manipulated By Non-Robust Features?cited by 12
  24. 2022Trap and Replace: Defending Backdoor Attacks by Trapping Them into an Easy-to-Replace Subnetworkcited by 12
  25. 2022Defending Against Disinformation Attacks in Open-Domain Question Answeringcited by 12
  26. 2021Sample Efficient Detection and Classification of Adversarial Attacks via Self-Supervised Embeddingscited by 11
  27. 2025Robustness and Cybersecurity in the EU Artificial Intelligence Actcited by 11
  28. 2025MTL-UE: Learning to Learn Nothing for Multi-Task Learningcited by 11
  29. 2023Sharpness-Aware Data Poisoning Attackcited by 10
  30. 2024SAP: Corrective Machine Unlearning with Scaled Activation Projection for Label Noise Robustnesscited by 10
  31. 2021Check Your Other Door! Creating Backdoor Attacks in the Frequency Domaincited by 9
  32. 2024Ungeneralizable Examplescited by 9
  33. 2025Context Reasoner: Incentivizing Reasoning Capability for Contextualized Privacy and Safety Compliance via Reinforcement Learningcited by 8
  34. 2021Disrupting Model Training with Adversarial Shortcutscited by 7
  35. 2023FreeEagle: Detecting Complex Neural Trojans in Data-Free Casescited by 7
  36. 2025No, of Course I Can! Deeper Fine-Tuning Attacks That Bypass Token-Level Safety Mechanismscited by 6
  37. 2021Incompatibility Clustering as a Defense Against Backdoor Poisoning Attackscited by 5
  38. 2022FairRoad: Achieving Fairness for Recommender Systems with Optimized Antidote Datacited by 5

50 most cited of 116 All 116 (JSON)