mlsanityData to 7 Oct 2026

Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning

arXiv 2205.01992 ↗First posted 4 May 2022cs.LG cs.AI cs.CRcited by 93

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

The success of machine learning is fueled by the increasing availability of computing power and large training datasets. The training data is used to learn new models or update existing ones, assuming that it is sufficiently representative of the data that will be encountered at test time. This assumption is challenged by the threat of poisoning, an attack that manipulates the training data to compromise the model's performance at test time. Although poisoning has been acknowledged as a relevant threat in industry applications, and a variety of different attacks and defenses have been proposed so far, a complete systematization and critical review of the field is still missing. In this survey, we provide a comprehensive systematization of poisoning attacks and defenses in machine learning, reviewing more than 100 papers published in the field in the last 15 years. We start by categorizing the current threat models and attacks, and then organize existing defenses accordingly. While we focus mostly on computer-vision applications, we argue that our systematization also encompasses state-of-the-art attacks and defenses for other data modalities. Finally, we discuss existing resources for research in poisoning, and shed light on the current limitations and open research questions in this research field.

Key facts. Cited by 93 papers in the map (data through 7 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 134 of its 214 references that are in the map. First posted on arXiv 4 May 2022.

Builds on

134 of 214 references.

Closest ancestors · 6

Same direction, newest first

  1. 2022Machine Learning Security against Data Poisoning: Are We There Yet?12 Apr
  2. 2022Improved Certified Defenses against Data Poisoning with (Deterministic) Finite Aggregation5 Feb
  3. 2021Trigger Hunting with a Topological Prior for Trojan Detection15 Oct
  4. 2021Sleeper Agent: Scalable Hidden Trigger Backdoors for Neural Networks Trained from Scratch16 Jun
  5. 2021Poisoning Deep Reinforcement Learning Agents with In-Distribution Triggers14 Jun
  6. 2021The Hammer and the Nut: Is Bilevel Optimization Really Needed to Poison Linear Classifiers?23 Mar

Other references · 128

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  3. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  4. 2018How To Backdoor Federated Learningcited by 745
  5. 2016Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimizationcited by 688
  6. 2017ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Modelscited by 656
  7. 2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 550
  8. 2018Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networkscited by 500
  9. 2018Spectral Signatures in Backdoor Attackscited by 457
  10. 2019On Evaluating Adversarial Robustnesscited by 442
  11. 2019STRIP: A Defence Against Trojan Attacks on Deep Neural Networkscited by 437
  12. 2018Detecting Backdoor Attacks on Deep Neural Networks by Activation Clusteringcited by 429
38 more
  1. 2020Backdoor Learning: A Surveycited by 412
  2. 2017Wild Patterns: Ten Years After the Rise of Adversarial Machine Learningcited by 411
  3. 2018Bilevel Programming for Hyperparameter Optimization and Meta-Learningcited by 403
  4. 2019Hidden Trigger Backdoor Attackscited by 359
  5. 2015Gradient-based Hyperparameter Optimization through Reversible Learningcited by 357
  6. 2018Analyzing Federated Learning through an Adversarial Lenscited by 346
  7. 2020Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networkscited by 341
  8. 2020Invisible Backdoor Attack with Sample-Specific Triggerscited by 338
  9. 2021Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networkscited by 313
  10. 2020Input-Aware Dynamic Backdoor Attackcited by 304
  11. 2019Optimizing Millions of Hyperparameters by Implicit Differentiationcited by 303
  12. 2017Certified Defenses for Data Poisoning Attackscited by 288
  13. 2020Data Poisoning Attacks Against Federated Learning Systemscited by 266
  14. 2019Adversarial Attacks on Graph Neural Networks via Meta Learningcited by 261
  15. 2018Adversarial Attacks and Defences: A Surveycited by 259
  16. 2021Anti-Backdoor Learning: Training Clean Models on Poisoned Datacited by 258
  17. 2019Can You Really Backdoor Federated Learning?cited by 250
  18. 2019A new Backdoor Attack in CNNs by training set corruption without label poisoningcited by 237
  19. 2020ONION: A Simple and Effective Defense Against Textual Backdoor Attackscited by 231
  20. 2021Adversarial Neuron Pruning Purifies Backdoored Deep Modelscited by 230
  21. 2018Adversarial Robustness Toolbox v1.0.0cited by 214
  22. 2019Detecting AI Trojans Using Meta Neural Analysiscited by 192
  23. 2017Towards Poisoning of Deep Learning Algorithms with Back-gradient Optimizationcited by 188
  24. 2020Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defensescited by 171
  25. 2018Adversarial Attack and Defense on Graph Data: A Surveycited by 166
  26. 2020Clean-Label Backdoor Attacks on Video Recognition Modelscited by 162
  27. 2021Adversarial Unlearning of Backdoors via Implicit Hypergradientcited by 160
  28. 2020Witches' Brew: Industrial Scale Data Poisoning via Gradient Matchingcited by 156
  29. 2017Neural Trojanscited by 151
  30. 2022Backdoor Defense via Decoupling the Training Processcited by 149
  31. 2018SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systemscited by 148
  32. 2019Universal Litmus Patterns: Revealing Backdoor Attacks in CNNscited by 143
  33. 2018Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbationcited by 131
  34. 2020Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustnesscited by 130
  35. 2019Transferable Clean-Label Poisoning Attacks on Deep Neural Netscited by 129
  36. 2020Backdoor Attacks Against Deep Learning Systems in the Physical Worldcited by 129
  37. 2020Backdoor Attacks to Graph Neural Networkscited by 126
  38. 2018Stronger Data Poisoning Attacks Break Data Sanitization Defensescited by 125

50 most cited of 128 All 128 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 24; 2024: 24; 2025: 20; 2026: 21.

93 papers.

Sort:

By area:

  1. 2023Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injectioncited by 143
  2. 2023On the Exploitability of Instruction Tuningcited by 110
  3. 2023Adversarial attacks and defenses in explainable artificial intelligence: A surveycited by 47
  4. 2022Hidden Poison: Machine Unlearning Enables Camouflaged Poisoning Attackscited by 37
  5. 2024Machine Unlearning Fails to Remove Data Poisoning Attackscited by 35
  6. 2022Machine Learning Security in Industry: A Quantitative Surveycited by 21
  7. 2024LLM-Assisted Crisis Management: Building Advanced LLM Platforms for Effective Emergency Response and Public Collaborationcited by 20
  8. 2025Data Poisoning in Deep Learning: A Surveycited by 20
  9. 2023Towards more Practical Threat Models in Artificial Intelligence Securitycited by 17
  10. 2026Intelligent AI Delegationcited by 16
  11. 2024Preference Poisoning Attacks on Reward Model Learningcited by 14
  12. 2024Human-inspired Perspectives: A Survey on AI Long-term Memorycited by 12
38 more
  1. 2023It Is All About Data: A Survey on the Effects of Data on Adversarial Robustnesscited by 9
  2. 2024Imperio: Language-Guided Backdoor Attacks for Arbitrary Model Controlcited by 8
  3. 2024Characterizing and Evaluating the Reliability of LLMs against Jailbreak Attackscited by 8
  4. 2025A Systematic Review of Poisoning Attacks Against Large Language Modelscited by 8
  5. 2022Thinking Two Moves Ahead: Anticipating Other Users Improves Backdoor Attacks in Federated Learningcited by 7
  6. 2023Universal Backdoor Attackscited by 7
  7. 2024U Can't Gen This? A Survey of Intellectual Property Protection Methods for Data in Generative AIcited by 7
  8. 2023Beyond Boundaries: A Comprehensive Survey of Transferable Attacks on AI Systemscited by 6
  9. 2025Statistical Collusion by Collectives on Learning Platformscited by 6
  10. 2023On the Security Risks of Knowledge Graph Reasoningcited by 5
  11. 2023A Proxy Attack-Free Strategy for Practically Improving the Poisoning Efficiency in Backdoor Attackscited by 5
  12. 2024A Backdoor Approach with Inverted Labels Using Dirty Label-Flipping Attackscited by 4
  13. 2024Data Poisoning Attacks in Intelligent Transportation Systems: A Surveycited by 4
  14. 2023Minimizing Energy Consumption of Deep Learning Models by Energy-Aware Trainingcited by 3
  15. 2023Fast Adversarial Label-Flipping Attack on Tabular Datacited by 3
  16. 2023A Red Teaming Framework for Securing AI in Maritime Autonomous Systemscited by 3
  17. 2024The Art of Deception: Robust Backdoor Attack using Dynamic Stacking of Triggerscited by 3
  18. 2024Testing autonomous vehicles and AI: perspectives and challenges from cybersecurity, transparency, robustness and fairnesscited by 3
  19. 2025Poisoning Bayesian Inference via Data Deletion and Replicationcited by 3
  20. 2022On the Robustness of Random Forest Against Untargeted Data Poisoning: An Ensemble-Based Approachcited by 2
  21. 2023Enhancement attacks in biomedical machine learningcited by 2
  22. 2023Dropout Attackscited by 2
  23. 2023Hardening RGB-D Object Recognition Systems against Adversarial Patch Attackscited by 2
  24. 2024On the Robustness of Adversarial Training Against Uncertainty Attackscited by 2
  25. 2024Edge-Only Universal Adversarial Attacks in Distributed Learningcited by 2
  26. 2024The Broader Landscape of Robustness in Algorithmic Statisticscited by 2
  27. 2024Emerging Security Challenges of Large Language Modelscited by 2
  28. 2025Buffer-free Class-Incremental Learning with Out-of-Distribution Detectioncited by 2
  29. 2025Scam2Prompt: A Scalable Framework for Auditing Malicious Scam Endpoints in Production LLMscited by 2
  30. 2025HMARK: Radioactive Multi-Bit Semantic-Latent Watermarking for Diffusion Modelscited by 2
  31. 2023On the Limitations of Model Stealing with Uncertainty Quantification Modelscited by 1
  32. 2023Quantifying the robustness of deep multispectral segmentation models against natural perturbations and data poisoningcited by 1
  33. 2024Robust Synthetic Data-Driven Detection of Living-Off-the-Land Reverse Shellscited by 1
  34. 2024Have You Poisoned My Data? Defending Neural Networks against Data Poisoningcited by 1
  35. 2024A Novel Perturb-ability Score to Mitigate Evasion Adversarial Attacks on Flow-Based ML-NIDScited by 1
  36. 2024Generalizable Targeted Data Poisoning against Varying Physical Objectscited by 1
  37. 2025Trust Under Siege: Label Spoofing Attacks against Machine Learning for Android Malware Detectioncited by 1
  38. 2025Stress-Testing ML Pipelines with Adversarial Data Corruptioncited by 1

50 most cited of 93 All 93 (JSON)