mlsanityData to 7 Oct 2026

Poisoning Web-Scale Training Datasets is Practical

arXiv 2302.10149 ↗First posted 20 Feb 2023cs.CR cs.LGcited by 276

Not on the map. Why?

Cited by 108 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Deep learning models are often trained on distributed, web-scale datasets crawled from the internet. In this paper, we introduce two new dataset poisoning attacks that intentionally introduce malicious examples to a model's performance. Our attacks are immediately practical and could, today, poison 10 popular datasets. Our first attack, split-view poisoning, exploits the mutable nature of internet content to ensure a dataset annotator's initial view of the dataset differs from the view downloaded by subsequent clients. By exploiting specific invalid trust assumptions, we show how we could have poisoned 0.01% of the LAION-400M or COYO-700M datasets for just 60 USD. Our second attack, frontrunning poisoning, targets web-scale datasets that periodically snapshot crowd-sourced content -- such as Wikipedia -- where an attacker only needs a time-limited window to inject malicious examples. In light of both attacks, we notify the maintainers of each affected dataset and recommended several low-overhead defenses.

Key facts. Cited by 276 papers in the map (data through 7 Oct 2026). Builds on 73 of its 141 references that are in the map. First posted on arXiv 20 Feb 2023.

Builds on

73 of 141 references.

References · 73

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2020Scaling Laws for Neural Language Modelscited by 6,411
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  7. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  8. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  9. 2016Pointer Sentinel Mixture Modelscited by 2,992
  10. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  11. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  12. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
38 more
  1. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  2. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
  3. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  4. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  5. 2017VGGFace2: A dataset for recognising faces across pose and agecited by 911
  6. 2021FLAVA: A Foundational Language And Vision Alignment Modelcited by 646
  7. 2020VGGSound: A Large-scale Audio-Visual Datasetcited by 641
  8. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 544
  9. 2018Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networkscited by 500
  10. 2018Spectral Signatures in Backdoor Attackscited by 457
  11. 2019STRIP: A Defence Against Trojan Attacks on Deep Neural Networkscited by 437
  12. 2018Detecting Backdoor Attacks on Deep Neural Networks by Activation Clusteringcited by 429
  13. 2018The Web as a Knowledge-base for Answering Complex Questionscited by 409
  14. 2019Hidden Trigger Backdoor Attackscited by 359
  15. 2020Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networkscited by 341
  16. 2020Invisible Backdoor Attack with Sample-Specific Triggerscited by 338
  17. 2021Multimodal datasets: misogyny, pornography, and malignant stereotypescited by 326
  18. 2020Attack of the Tails: Yes, You Really Can Backdoor Federated Learningcited by 307
  19. 2020Input-Aware Dynamic Backdoor Attackcited by 304
  20. 2019Label-Consistent Backdoor Attackscited by 285
  21. 2021Anti-Backdoor Learning: Training Clean Models on Poisoned Datacited by 258
  22. 2022Red-Teaming the Stable Diffusion Safety Filtercited by 257
  23. 2019A new Backdoor Attack in CNNs by training set corruption without label poisoningcited by 237
  24. 2015YFCC100M: The New Data in Multimedia Researchcited by 234
  25. 2021Adversarial Neuron Pruning Purifies Backdoored Deep Modelscited by 230
  26. 2018Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learningcited by 219
  27. 2021Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Triggercited by 203
  28. 2015The MegaFace Benchmark: 1 Million Faces for Recognition at Scalecited by 199
  29. 2019Detecting AI Trojans Using Meta Neural Analysiscited by 192
  30. 2017Towards Poisoning of Deep Learning Algorithms with Back-gradient Optimizationcited by 188
  31. 2019CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEBcited by 177
  32. 2017Deep Learning is Robust to Massive Label Noisecited by 164
  33. 2020Witches' Brew: Industrial Scale Data Poisoning via Gradient Matchingcited by 156
  34. 2021Poisoning and Backdooring Contrastive Learningcited by 149
  35. 2020Dynamic Backdoor Attacks Against Machine Learning Modelscited by 147
  36. 2020REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasetscited by 136
  37. 2018Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbationcited by 131
  38. 2015Large-scale Simple Question Answering with Memory Networkscited by 127

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 40; 2024: 83; 2025: 81; 2026: 72.

276 papers.

Sort:

By area:

  1. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
  2. 2023DataComp: In search of the next generation of multimodal datasetscited by 635
  3. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 478
  4. 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Basescited by 402
  5. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
  6. 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 343
  7. 2024Knowledge Conflicts for LLMs: A Surveycited by 274
  8. 2023Provable Robust Watermarking for AI-Generated Textcited by 259
  9. 2023Challenges and Applications of Large Language Modelscited by 251
  10. 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Modelscited by 251
  11. 2024Rethinking Machine Unlearning for Large Language Modelscited by 247
  12. 2024Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Modelscited by 158
38 more
  1. 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMscited by 158
  2. 2025Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMscited by 158
  3. 2024AI Sandbagging: Language Models can Strategically Underperform on Evaluationscited by 131
  4. 2024The Emerged Security and Privacy of LLM Agent: A Survey with Case Studiescited by 127
  5. 2023On the Risk of Misinformation Pollution with Large Language Modelscited by 124
  6. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
  7. 2023On the Exploitability of Instruction Tuningcited by 110
  8. 2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 91
  9. 2023Identifying and Mitigating the Security Risks of Generative AIcited by 90
  10. 2024Defending Against Unforeseen Failure Modes with Latent Adversarial Trainingcited by 82
  11. 2026Skill-Inject: Measuring Agent Vulnerability to Skill File Attackscited by 82
  12. 2023Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilitiescited by 76
  13. 2023Nightshade: Prompt-Specific Poisoning Attacks on Text-to-Image Generative Modelscited by 72
  14. 2025Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samplescited by 66
  15. 2023CleanCLIP: Mitigating Data Poisoning Attacks in Multimodal Contrastive Learningcited by 62
  16. 2024Exploiting LLM Quantizationcited by 61
  17. 2024Shadowcast: Stealthy Data Poisoning Attacks Against Vision-Language Modelscited by 48
  18. 2024Phantom: General Backdoor Attacks on Retrieval Augmented Language Generationcited by 46
  19. 2024Persistent Pre-Training Poisoning of LLMscited by 46
  20. 2024Consent in Crisis: The Rapid Decline of the AI Data Commonscited by 45
  21. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
  22. 2023Training Data Extraction From Pre-trained Language Models: A Surveycited by 43
  23. 2026Supply-Chain Poisoning Attacks Against LLM Coding Agent Skill Ecosystemscited by 42
  24. 2023Leveraging Vision-Language Models for Improving Domain Generalization in Image Classificationcited by 41
  25. 2025Ctrl-Z: Controlling AI Agents via Resamplingcited by 40
  26. 2023SoK: Memorization in General-Purpose Large Language Modelscited by 36
  27. 2024LLMs Meet Multimodal Generation and Editing: A Surveycited by 35
  28. 2025Practical Poisoning Attacks against Retrieval-Augmented Generationcited by 35
  29. 2024The Stronger the Diffusion Model, the Easier the Backdoor: Data Poisoning to Induce Copyright Breaches Without Adjusting Finetuning Pipelinecited by 33
  30. 2024Best-of-Venom: Attacking RLHF by Injecting Poisoned Preference Datacited by 33
  31. 2025Topic-FlipRAG: Topic-Orientated Adversarial Opinion Manipulation Attacks to Retrieval-Augmented Generation Modelscited by 32
  32. 2023Ecosystem Graphs: The Social Footprint of Foundation Modelscited by 31
  33. 2024PoisonBench: Assessing Large Language Model Vulnerability to Data Poisoningcited by 31
  34. 2023Towards Stable Backdoor Purification through Feature Shift Tuningcited by 30
  35. 2024Teach LLMs to Phish: Stealing Private Information from Language Modelscited by 29
  36. 2024CROW: Eliminating Backdoors from Large Language Models via Internal Consistency Regularizationcited by 29
  37. 2025PR-Attack: Coordinated Prompt-RAG Attacks on Retrieval-Augmented Generation in Large Language Models via Bilevel Optimizationcited by 29
  38. 2023Towards Possibilities & Impossibilities of AI-generated Text Detection: A Surveycited by 28

50 most cited of 276 All 276 (JSON)