mlsanityData to 7 Oct 2026

Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses

arXiv 2012.10544 ↗First posted 18 Dec 2020cs.LG cs.AI cs.CRcited by 171

Not on the map. Why?

Cited by 60 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

As machine learning systems grow in scale, so do their training data requirements, forcing practitioners to automate and outsource the curation of training data in order to achieve state-of-the-art performance. The absence of trustworthy human supervision over the data collection process exposes organizations to security vulnerabilities; training data can be manipulated to control and degrade the downstream behaviors of learned models. The goal of this work is to systematically categorize and discuss a wide range of dataset vulnerabilities and exploits, approaches for defending against these threats, and an array of open problems in this space. In addition to describing various poisoning and backdoor threat models and the relationships among them, we develop their unified taxonomy.

Key facts. Cited by 171 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 28; 2023: 49; 2024: 30; 2025: 33; 2026: 19.

171 papers.

Sort:

By area:

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  2. 2023Boundary Unlearningcited by 132
  3. 2021Back to the Drawing Board: A Critical Evaluation of Poisoning Attacks on Production Federated Learningcited by 116
  4. 2021Adversarial Examples Make Strong Poisonscited by 115
  5. 2021Sleeper Agent: Scalable Hidden Trigger Backdoors for Neural Networks Trained from Scratchcited by 98
  6. 2022Neurotoxin: Durable Backdoors in Federated Learningcited by 97
  7. 2022Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoningcited by 93
  8. 2023Diffusion Models for Imperceptible and Transferable Adversarial Attackcited by 93
  9. 2021BadPre: Task-agnostic Backdoor Attacks to Pre-trained NLP Foundation Modelscited by 76
  10. 2024BadChain: Backdoor Chain-of-Thought Prompting for Large Language Modelscited by 74
  11. 2023Nightshade: Prompt-Specific Poisoning Attacks on Text-to-Image Generative Modelscited by 72
  12. 2023Unleashing the potential of prompt engineering for large language modelscited by 72
38 more
  1. 2024Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agentscited by 63
  2. 2021Availability Attacks Create Shortcutscited by 60
  3. 2021Better Safe Than Sorry: Preventing Delusive Adversaries with Adversarial Trainingcited by 58
  4. 2022Improved Certified Defenses against Data Poisoning with (Deterministic) Finite Aggregationcited by 50
  5. 2025Commercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attackscited by 49
  6. 2022Autoregressive Perturbations for Data Poisoningcited by 45
  7. 2023An Empirical Study of Pre-Trained Model Reuse in the Hugging Face Deep Learning Model Registrycited by 45
  8. 2023Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomycited by 44
  9. 2023Personalization as a Shortcut for Few-Shot Backdoor Attack against Text-to-Image Diffusion Modelscited by 42
  10. 2022Hidden Poison: Machine Unlearning Enables Camouflaged Poisoning Attackscited by 37
  11. 2021Preventing Unauthorized Use of Proprietary Data: Poisoning for Secure Dataset Releasecited by 35
  12. 2024Machine Unlearning Fails to Remove Data Poisoning Attackscited by 35
  13. 2021DP-InstaHide: Provably Defusing Poisoning and Backdoor Attacks with Differentially Private Data Augmentationscited by 33
  14. 2023Backdoor Attacks and Defenses in Federated Learning: Survey, Challenges and Future Research Directionscited by 33
  15. 2023Don't trust your eyes: on the (un)reliability of feature visualizationscited by 32
  16. 2023Towards Stable Backdoor Purification through Feature Shift Tuningcited by 30
  17. 2022Energy-Latency Attacks via Sponge Poisoningcited by 29
  18. 2023Vulnerabilities in AI Code Generators: Exploring Targeted Data Poisoning Attackscited by 29
  19. 2022Untargeted Backdoor Attack against Object Detectioncited by 28
  20. 2023Towards Stealthy Backdoor Attacks against Speech Recognition via Elements of Soundcited by 28
  21. 2024Corrective Machine Unlearningcited by 26
  22. 2023BackdoorBox: A Python Toolbox for Backdoor Learningcited by 25
  23. 2026BadSkill: Backdoor Attacks on Agent Skills via Model-in-Skill Poisoningcited by 25
  24. 2022Indiscriminate Data Poisoning Attacks on Neural Networkscited by 24
  25. 2025Antidistillation Samplingcited by 24
  26. 2023Exploring the Limits of Model-Targeted Indiscriminate Data Poisoning Attackscited by 23
  27. 2021What Doesn't Kill You Makes You Robust(er): How to Adversarially Train against Data Poisoningcited by 22
  28. 2023Backdoor Attacks Against Dataset Distillationcited by 20
  29. 2025Data Poisoning in Deep Learning: A Surveycited by 20
  30. 2023The Devil's Advocate: Shattering the Illusion of Unexploitable Data using Diffusion Modelscited by 19
  31. 2024Purify Unlearnable Examples via Rate-Constrained Variational Autoencoderscited by 18
  32. 2022Holistic Robust Data-Driven Decisionscited by 17
  33. 2022How to Sift Out a Clean Data Subset in the Presence of Data Poisoning?cited by 17
  34. 2023Data and Model Poisoning Backdoor Attacks on Wireless Federated Learning, and the Defense Mechanisms: A Comprehensive Surveycited by 17
  35. 2024AI Risk Management Should Incorporate Both Safety and Securitycited by 17
  36. 2023Revisiting Personalized Federated Learning: Robustness Against Backdoor Attackscited by 16
  37. 2023Performance Scaling via Optimal Transport: Enabling Data Selection from Partially Revealed Sourcescited by 16
  38. 2025Joint-GCG: Unified Gradient-Based Poisoning Attacks on Retrieval-Augmented Generation Systemscited by 16

50 most cited of 171 All 171 (JSON)