Data to 14 Oct 2026

Poison Forensics: Traceback of Data Poisoning Attacks in Neural Networks

arXiv 2110.06904 ↗First posted 13 Oct 2021cs.CR cs.AIcited by 40

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

In adversarial machine learning, new defenses against attacks on deep learning systems are routinely broken soon after their release by more powerful attacks. In this context, forensic tools can offer a valuable complement to existing defenses, by tracing back a successful attack to its root cause, and offering a path forward for mitigation to prevent similar attacks in the future. In this paper, we describe our efforts in developing a forensic traceback tool for poison attacks on deep neural networks. We propose a novel iterative clustering and pruning solution that trims "innocent" training samples, until all that remains is the set of poisoned data responsible for the attack. Our method clusters training samples based on their impact on model parameters, then uses an efficient data unlearning method to prune innocent clusters. We empirically demonstrate the efficacy of our system on three types of dirty-label (backdoor) poison attacks and three types of clean-label poison attacks, across domains of computer vision and malware classification. Our system achieves over 98.4% precision and 96.8% recall across all attacks. We also show that our system is robust against four anti-forensics measures specifically designed to attack it.

Key facts. Cited by 40 papers in the map (data through 14 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 42 of its 90 references that are in the map. First posted on arXiv 13 Oct 2021.

Builds on

42 of 90 references.

Closest ancestors · 1

Same direction, newest first

2021Manipulating SGD with Data Ordering Attacks19 Apr

Other references · 41

2015Deep Residual Learning for Image Recognitioncited by 52,096
2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
2016Wide Residual Networkscited by 3,070
2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
2017Understanding Black-box Predictions via Influence Functionscited by 1,741
2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,512
2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,268
2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,106
2019Machine Unlearningcited by 1,061
2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 580
2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 546

29 more

2019Certified Data Removal from Machine Learning Modelscited by 535
2018Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networkscited by 519
2018Spectral Signatures in Backdoor Attackscited by 484
2017Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Modelscited by 462
2019STRIP: A Defence Against Trojan Attacks on Deep Neural Networkscited by 459
2018Detecting Backdoor Attacks on Deep Neural Networks by Activation Clusteringcited by 456
2019Hidden Trigger Backdoor Attackscited by 371
2019HopSkipJumpAttack: A Query-Efficient Decision-Based Attackcited by 333
2020Descent-to-Delete: Gradient-Based Methods for Machine Unlearningcited by 311
2020Influence Functions in Deep Learning Are Fragilecited by 252
2020Blind Backdoors in Deep Learning Modelscited by 182
2018EMBER: An Open Dataset for Training Static PE Malware Machine Learning Modelscited by 165
2020Witches' Brew: Industrial Scale Data Poisoning via Gradient Matchingcited by 163
2018Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbationcited by 134
2020Backdoor Attacks Against Deep Learning Systems in the Physical Worldcited by 134
2019Transferable Clean-Label Poisoning Attacks on Deep Neural Netscited by 131
2020You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completioncited by 131
2018Out-of-Distribution Detection Using an Ensemble of Self Supervised Leave-out Classifierscited by 126
2019Demon in the Variant: Statistical Analysis of DNNs for Robust Backdoor Contamination Detectioncited by 124
2020Adversarial Machine Learning -- Industry Perspectivescited by 122
2020MetaPoison: Practical General-purpose Clean-label Data Poisoningcited by 120
2020Intrinsic Certified Robustness of Bagging against Data Poisoning Attackscited by 94
2020Strong Data Augmentation Sanitizes Poisoning and Backdoor Attacks Without an Accuracy Tradeoffcited by 89
2020On Certifying Robustness against Backdoor Attacks via Randomized Smoothingcited by 77
2020Bullseye Polytope: A Scalable Clean-Label Poisoning Attack with Improved Transferabilitycited by 66
2021Handcrafted Backdoors in Deep Neural Networkscited by 63
2016Defensive Distillation is Not Robust to Adversarial Examplescited by 60
2021Poisoning the Unlabeled Dataset of Semi-Supervised Learningcited by 44
2019Gotta Catch 'Em All: Using Honeypots to Catch Adversarial Attacks on Neural Networkscited by 40

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 9; 2024: 13; 2025: 10; 2026: 5.

40 papers.

Sort:

By area:

2023Poisoning Web-Scale Training Datasets is Practicalcited by 290
2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Modelscited by 267
2024What is in Your Safe Data? Identifying Benign Data that Breaks Safetycited by 118
2023Machine Unlearning: Solutions and Challengescited by 93
2023Nightshade: Prompt-Specific Poisoning Attacks on Text-to-Image Generative Modelscited by 75
2022"Real Attackers Don't Compute Gradients": Bridging the Gap Between Adversarial ML Research and Practicecited by 56
2022MM-BD: Post-Training Detection of Backdoor Attacks with Arbitrary Backdoor Pattern Types Using a Maximum Margin Statisticcited by 55
2024Machine Unlearning Fails to Remove Data Poisoning Attackscited by 37
2023Vulnerabilities in AI Code Generators: Exploring Targeted Data Poisoning Attackscited by 33
2025Traceback of Poisoning Attacks to Retrieval-Augmented Generationcited by 28
2022Just Rotate it: Deploying Backdoor Attacks via Rotation Transformationcited by 20
2023BEAGLE: Forensics of Deep Learning Backdoor Attack for Better Defensecited by 15

28 more

2025TracLLM: A Generic Framework for Attributing Long Context LLMscited by 13
2025Who Taught the Lie? Responsibility Attribution for Poisoned Knowledge in Retrieval-Augmented Generationcited by 12
2023Temporal Robustness against Data Poisoningcited by 10
2025Data Duplication: A Novel Multi-Purpose Attack Paradigm in Machine Unlearningcited by 9
2025DATABench: Evaluating Dataset Auditing in Deep Learning from an Adversarial Perspectivecited by 8
2025Position: Bridge the Gaps between Machine Unlearning and AI Regulationcited by 7
2025AttnTrace: Contextual Attribution of Prompt Injection and Knowledge Corruptioncited by 7
2024Tracing Back the Malicious Clients in Poisoning Attacks to Federated Learningcited by 6
2023Universal Detection of Backdoor Attacks via Density-based Clustering and Centroids Analysiscited by 5
2024Mudjacking: Patching Backdoor Vulnerabilities in Foundation Modelscited by 4
2026Beware What You Autocomplete: Forensic Attribution of Backdoored Code Completionscited by 4
2024FCert: Certifiably Robust Few-Shot Classification in the Era of Foundation Modelscited by 3
2024UTrace: Poisoning Forensics for Private Collaborative Learningcited by 3
2024Towards Scalable and Robust Model Versioningcited by 2
2024Securing Voice Authentication Applications Against Targeted Data Poisoningcited by 2
2026Patcher: Post-Hoc Patching of Backdoored Large Language Modelscited by 2
2026Who Broke the System? Failure Localization in LLM-Based Multi-Agent Systemscited by 2
2023Pick your Poison: Undetectability versus Robustness in Data Poisoning Attackscited by 1
2024AI Horizon Scanning, White Paper p3395, IEEE-SA. Part I: Areas of Attentioncited by 1
2024Towards Assurance of LLM Adversarial Robustness using Ontology-Driven Argumentationcited by 1
2024Taking off the Rose-Tinted Glasses: A Critical Look at Adversarial ML Through the Lens of Evasion Attackscited by 1
2025Trust Under Siege: Label Spoofing Attacks against Machine Learning for Android Malware Detectioncited by 1
2025Detecting Stealthy Data Poisoning Attacks in AI Code Generatorscited by 1
2023SEA: Shareable and Explainable Attribution for Query-based Black-box Attackscited by 0
2024Binary Linear Tree Commitment-based Ownership Protection for Distributed Machine Learningcited by 0
2025Like Oil and Water: Group Robustness Methods and Poisoning Defenses May Be at Oddscited by 0
2026TriShieldRAG: 3 Rings, One Blind Spot in Layered Defenses for Retrieval-Augmented Generationcited by 0
2026TraceGuard: Adaptive Multimodal Poison Filtering through Cross-Feature Rank Agreementcited by 0

Similar, not cited

Close in wording, no citation either way