Data to 14 Oct 2026

Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering

arXiv 1811.03728 ↗First posted 9 Nov 2018cs.LG cs.CR stat.MLcited by 456

Not on the map. Why?

Cited by 145 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

While machine learning (ML) models are being increasingly trusted to make decisions in different and varying areas, the safety of systems using such models has become an increasing concern. In particular, ML models are often trained on data from potentially untrustworthy sources, providing adversaries with the opportunity to manipulate them by inserting carefully crafted samples into the training set. Recent work has shown that this type of attack, called a poisoning attack, allows adversaries to insert backdoors or trojans into the model, enabling malicious behavior with simple external backdoor triggers at inference time and only a blackbox perspective of the model itself. Detecting this type of attack is challenging because the unexpected behavior occurs only when a backdoor trigger, which is known only to the adversary, is present. Model users, either direct users of training data or users of pre-trained model from a catalog, may not guarantee the safe operation of their ML-based system. In this paper, we propose a novel approach to backdoor detection and removal for neural networks. Through extensive experimental results, we demonstrate its effectiveness for neural networks classifying text and images. To the best of our knowledge, this is the first methodology capable of detecting poisonous data crafted to insert backdoors and repairing the model that does not require a verified and trusted dataset.

Key facts. Cited by 456 papers in the map (data through 14 Oct 2026). First posted on arXiv 9 Nov 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 64; 2022: 84; 2023: 94; 2024: 98; 2025: 61; 2026: 55.

456 papers.

Sort:

By area:

2021Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networkscited by 324
2023Poisoning Web-Scale Training Datasets is Practicalcited by 290
2021Anti-Backdoor Learning: Training Clean Models on Poisoned Datacited by 263
2024Security and Privacy Challenges of Large Language Models: A Surveycited by 186
2021Adversarial Unlearning of Backdoors via Implicit Hypergradientcited by 166
2021Rethinking the Backdoor Attacks' Triggers: A Frequency Perspectivecited by 161
2022Backdoor Defense via Decoupling the Training Processcited by 153
2021SPECTRE: Defending Against Backdoor Attacks Using Robust Statisticscited by 130
2022BackdoorBench: A Comprehensive Benchmark of Backdoor Learningcited by 128
2021BadEncoder: Backdoor Attacks to Pre-trained Encoders in Self-Supervised Learningcited by 123
2021Sleeper Agent: Scalable Hidden Trigger Backdoors for Neural Networks Trained from Scratchcited by 102
2021Backdoor Scanning for Deep Neural Networks through K-Arm Optimizationcited by 98

38 more

2023Backdoor Attacks for In-Context Learning with Language Modelscited by 96
2022Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoningcited by 94
2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 93
2022BppAttack: Stealthy and Efficient Trojan Attacks against Deep Neural Networks via Image Quantization and Contrastive Adversarial Learningcited by 86
2022Untargeted Backdoor Watermark: Towards Harmless and Stealthy Dataset Copyright Protectioncited by 86
2024BadChain: Backdoor Chain-of-Thought Prompting for Large Language Modelscited by 80
2023Nightshade: Prompt-Specific Poisoning Attacks on Text-to-Image Generative Modelscited by 75
2022A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarkscited by 74
2021Black-box Detection of Backdoor Attacks with Limited Information and Datacited by 73
2021Spinning Language Models: Risks of Propaganda-As-A-Service and Countermeasurescited by 73
2021BACKDOORL: Backdoor Attack against Competitive Reinforcement Learningcited by 68
2023Enhancing Fine-Tuning Based Backdoor Defense with Sharpness-Aware Minimizationcited by 64
2021Handcrafted Backdoors in Deep Neural Networkscited by 63
2022The "Beatrix'' Resurrections: Robust Backdoor Detection via Gram Matricescited by 63
2023CleanCLIP: Mitigating Data Poisoning Attacks in Multimodal Contrastive Learningcited by 63
2021T-Miner: A Generative Approach to Defend Against Trojan Attacks on DNN-based Text Classificationcited by 60
2023TrojanPuzzle: Covertly Poisoning Code-Suggestion Modelscited by 60
2021Better Safe Than Sorry: Preventing Delusive Adversaries with Adversarial Trainingcited by 59
2023Reconstructive Neuron Pruning for Backdoor Defensecited by 59
2022Fine-mixing: Mitigating Backdoors in Fine-tuned Language Modelscited by 57
2022MM-BD: Post-Training Detection of Backdoor Attacks with Arbitrary Backdoor Pattern Types Using a Maximum Margin Statisticcited by 55
2021AEVA: Black-box Backdoor Detection Using Adversarial Extreme Value Analysiscited by 53
2023Privacy in Large Language Models: Attacks, Defenses and Future Directionscited by 51
2021Poison Ink: Robust and Invisible Backdoor Attackcited by 50
2022Measuring the Effect of Training Data on Deep Learning Predictions via Randomized Experimentscited by 50
2023Stealthy Backdoor Attack for Code Modelscited by 49
2022Beyond Explaining: Opportunities and Challenges of XAI-Based Model Improvementcited by 48
2021CoProtector: Protect Open-Source Code against Unauthorized Training Usage with Data Poisoningcited by 47
2023Detecting Backdoors During the Inference Stage Based on Corruption Robustness Consistencycited by 46
2023UNICORN: A Unified Backdoor Trigger Inversion Frameworkcited by 46
2024Stress-Testing Capability Elicitation With Password-Locked Modelscited by 46
2021Can You Hear It? Backdoor Attacks via Ultrasonic Triggerscited by 44
2021This looks more like that: Enhancing Self-Explaining Models by Prototypical Relevance Propagationcited by 43
2023Shared Adversarial Unlearning: Backdoor Mitigation by Unlearning Shared Adversarial Examplescited by 43
2021An Overview of Backdoor Attacks Against Deep Neural Networks and Possible Defencescited by 42
2021Poison Forensics: Traceback of Data Poisoning Attacks in Neural Networkscited by 40
2022Towards A Proactive ML Approach for Detecting Backdoor Poison Samplescited by 40
2023Privacy Side Channels in Machine Learning Systemscited by 40

50 most cited of 456 All 456 (JSON)