mlsanityData to 7 Oct 2026

Data Poisoning in Deep Learning: A Survey

arXiv 2503.22759 ↗First posted 27 Mar 2025cs.CR cs.AIcited by 20

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

Deep learning has become a cornerstone of modern artificial intelligence, enabling transformative applications across a wide range of domains. As the core element of deep learning, the quality and security of training data critically influence model performance and reliability. However, during the training process, deep learning models face the significant threat of data poisoning, where attackers introduce maliciously manipulated training data to degrade model accuracy or lead to anomalous behavior. While existing surveys provide valuable insights into data poisoning, they generally adopt a broad perspective, encompassing both attacks and defenses, but lack a dedicated, in-depth analysis of poisoning attacks specifically in deep learning. In this survey, we bridge this gap by presenting a comprehensive and targeted review of data poisoning in deep learning. First, this survey categorizes data poisoning attacks across multiple perspectives, providing an in-depth analysis of their characteristics and underlying design princinples. Second, the discussion is extended to the emerging area of data poisoning in large language models(LLMs). Finally, we explore critical open challenges in the field and propose potential research directions to advance the field further. To support further exploration, an up-to-date repository of resources on data poisoning in deep learning is available at https://github.com/Pinlong-Zhao/Data-Poisoning.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 102 of its 166 references that are in the map. First posted on arXiv 27 Mar 2025.

Builds on

102 of 166 references.

Closest ancestors · 17

Same direction, newest first

  1. 2024PoisonBench: Assessing Large Language Model Vulnerability to Data Poisoning11 Oct
  2. 2024Scaling Trends for Data Poisoning in LLMs6 Aug
  3. 2024Turning Generative Models Degenerate: The Power of Data Poisoning Attacks17 Jul
  4. 2024Shadowcast: Stealthy Data Poisoning Attacks Against Vision-Language Models5 Feb
  5. 2024Data Poisoning for In-context Learning3 Feb
  6. 2024Universal Vulnerabilities in Large Language Models: Backdoor Attacks for In-context Learning11 Jan
  7. 2023Label Poisoning is All You Need29 Oct
  8. 2023Nightshade: Prompt-Specific Poisoning Attacks on Text-to-Image Generative Models20 Oct
  9. 2023PoisonPrompt: Backdoor Attack on Prompt-based Large Language Models19 Oct
  10. 2023Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injection31 Jul
  11. 2023On the Exploitability of Instruction Tuning28 Jun
  12. 2023Instructions as Backdoors: Backdoor Vulnerabilities of Instruction Tuning for Large Language Models24 May
5 more
  1. 2023Stealthy Backdoor Attack for Code Models6 Jan
  2. 2022CorruptEncoder: Data Poisoning based Backdoor Attacks to Contrastive Learning15 Nov
  3. 2022Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning4 May
  4. 2022Machine Learning Security against Data Poisoning: Are We There Yet?12 Apr
  5. 2022Poisoning Attacks and Defenses on Artificial Intelligence: A Survey21 Feb

Other references · 85

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  8. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  9. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  10. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  11. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  12. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
38 more
  1. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  2. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
  3. 2018How To Backdoor Federated Learningcited by 745
  4. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
  5. 2023Instruction Tuning with GPT-4cited by 676
  6. 2024Phi-4 Technical Reportcited by 675
  7. 2017ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Modelscited by 656
  8. 2023Instruction Tuning for Large Language Models: A Surveycited by 617
  9. 2018Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networkscited by 500
  10. 2019Local Model Poisoning Attacks to Byzantine-Robust Federated Learningcited by 479
  11. 2018Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Surveycited by 459
  12. 2019Hidden Trigger Backdoor Attackscited by 359
  13. 2018Analyzing Federated Learning through an Adversarial Lenscited by 346
  14. 2020Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networkscited by 341
  15. 2020Invisible Backdoor Attack with Sample-Specific Triggerscited by 338
  16. 2023LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Modelscited by 317
  17. 2023PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimizationcited by 311
  18. 2020Input-Aware Dynamic Backdoor Attackcited by 304
  19. 2019Optimizing Millions of Hyperparameters by Implicit Differentiationcited by 303
  20. 2019Label-Consistent Backdoor Attackscited by 285
  21. 2024Long-context LLMs Struggle with Long In-context Learningcited by 285
  22. 2023Poisoning Web-Scale Training Datasets is Practicalcited by 276
  23. 2020Data Poisoning Attacks Against Federated Learning Systemscited by 266
  24. 2023Poisoning Language Models During Instruction Tuningcited by 254
  25. 2020Influence Functions in Deep Learning Are Fragilecited by 247
  26. 2019A new Backdoor Attack in CNNs by training set corruption without label poisoningcited by 237
  27. 2018Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learningcited by 219
  28. 2017Towards Poisoning of Deep Learning Algorithms with Back-gradient Optimizationcited by 188
  29. 2020Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defensescited by 171
  30. 2018Adversarial Attack and Defense on Graph Data: A Surveycited by 166
  31. 2020Witches' Brew: Industrial Scale Data Poisoning via Gradient Matchingcited by 156
  32. 2021Poisoning and Backdooring Contrastive Learningcited by 149
  33. 2020Dynamic Backdoor Attacks Against Machine Learning Modelscited by 147
  34. 2018Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbationcited by 131
  35. 2019Transferable Clean-Label Poisoning Attacks on Deep Neural Netscited by 129
  36. 2021Advances in adversarial attacks and defenses in computer vision: A surveycited by 127
  37. 2018Stronger Data Poisoning Attacks Break Data Sanitization Defensescited by 125
  38. 2023Universal Jailbreak Backdoors from Poisoned Human Feedbackcited by 120

50 most cited of 85 All 85 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 9; 2026: 11.

20 papers.

Sort:

By area:

  1. 2025A-MemGuard: A Proactive Defense Framework for LLM-Based Agent Memorycited by 61
  2. 2025Your Agent May Misevolve: Emergent Risks in Self-evolving LLM Agentscited by 52
  3. 2026Test-Time Attention Purification for Backdoored Large Vision Language Modelscited by 5
  4. 2026Safety, Security, and Cognitive Risks in World Modelscited by 3
  5. 2025Scam2Prompt: A Scalable Framework for Auditing Malicious Scam Endpoints in Production LLMscited by 2
  6. 2025FuncPoison: Poisoning Function Library to Hijack Multi-agent Autonomous Driving Systemscited by 2
  7. 2025ALMGuard: Safety Shortcuts and Where to Find Them as Guardrails for Audio-Language Modelscited by 2
  8. 2026When AI reviews science: Can we trust the referee?cited by 2
  9. 2025Superior resilience to poisoning and amenability to unlearning in quantum machine learningcited by 1
  10. 2025Elytra: A Flexible Framework for Securing Large Vision Systemscited by 0
  11. 2025Can Artificial Intelligence solve the blockchain oracle problem? Unpacking the Challenges and Possibilitiescited by 0
  12. 2025Bilevel Models for Adversarial Learning and A Case Studycited by 0
8 more
  1. 2026Topology-Independent Robustness of the Weighted Mean under Label Poisoning Attacks in Heterogeneous Decentralized Learningcited by 0
  2. 2026PoiCGAN: A Targeted Poisoning Based on Feature-Label Joint Perturbation in Federated Learningcited by 0
  3. 2026Hijacking Text Heritage: Hiding the Human Signature through Homoglyphic Substitutioncited by 0
  4. 2026On the Fragility of Data Attribution When Learning Is Distributedcited by 0
  5. 2026It Takes One to Bias Them All: Breaking Bad with One-Shot GRPOcited by 0
  6. 2026The Role of Rigor in Artificial Intelligencecited by 0
  7. 2026Empirical Evaluation of Data Poisoning Attacks in Supervised Learningcited by 0
  8. 2026Purifying Backdoored Large Vision-Language Models by Removing Hijacked Directionscited by 0