Trojan Horse Training for Breaking Defenses against Backdoor Attacks in Deep Learning
Not on the map. Why?
Abstract (arXiv)
Machine learning (ML) models that use deep neural networks are vulnerable to backdoor attacks. Such attacks involve the insertion of a (hidden) trigger by an adversary. As a consequence, any input that contains the trigger will cause the neural network to misclassify the input to a (single) target class, while classifying other inputs without a trigger correctly. ML models that contain a backdoor are called Trojan models. Backdoors can have severe consequences in safety-critical cyber and cyber physical systems when only the outputs of the model are available. Defense mechanisms have been developed and illustrated to be able to distinguish between outputs from a Trojan model and a non-Trojan model in the case of a single-target backdoor attack with accuracy > 96 percent. Understanding the limitations of a defense mechanism requires the construction of examples where the mechanism fails. Current single-target backdoor attacks require one trigger per target class. We introduce a new, more general attack that will enable a single trigger to result in misclassification to more than one target class. Such a misclassification will depend on the true (actual) class that the input belongs to. We term this category of attacks multi-target backdoor attacks. We demonstrate that a Trojan model with either a single-target or multi-target trigger can be trained so that the accuracy of a defense mechanism that seeks to distinguish between outputs coming from a Trojan and a non-Trojan model will be reduced. Our approach uses the non-Trojan model as a teacher for the Trojan model and solves a min-max optimization problem between the Trojan model and defense mechanism. Empirical evaluations demonstrate that our training procedure reduces the accuracy of a state-of-the-art defense mechanism from >96 to 0 percent.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 37 of its 52 references that are in the map. First posted on arXiv 25 Mar 2022.
Builds on
37 of 52 references.
References · 37
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2014Explaining and Harnessing Adversarial Examplescited by 7,364
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
- 2016Adversarial examples in the physical worldcited by 1,769
- 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,318
- 2016Universal adversarial perturbationscited by 784
- 2018How To Backdoor Federated Learningcited by 765
- 2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 580
- 2018Spectral Signatures in Backdoor Attackscited by 484
- 2018Detecting Backdoor Attacks on Deep Neural Networks by Activation Clusteringcited by 456
- 2018Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooringcited by 357
- 2020Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networkscited by 349
25 more
- 2021Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networkscited by 324
- 2019Label-Consistent Backdoor Attackscited by 296
- 2016Understanding Deep Neural Networks with Rectified Linear Unitscited by 257
- 2019A backdoor attack against LSTM-based text classification systemscited by 227
- 2017Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Modelscited by 206
- 2019Detecting AI Trojans Using Meta Neural Analysiscited by 199
- 2020Blind Backdoors in Deep Learning Modelscited by 182
- 2017Neural Trojanscited by 153
- 2019Universal Litmus Patterns: Revealing Backdoor Attacks in CNNscited by 146
- 2018Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbationcited by 134
- 2020You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completioncited by 131
- 2018Deep Learning in Mobile and Wireless Networking: A Surveycited by 118
- 2019TBT: Targeted Neural Network Attack with Bit Trojancited by 113
- 2019Robust Anomaly Detection and Backdoor Attack Detection Via Differential Privacycited by 95
- 2020Practical Detection of Trojan Neural Networks: Data-Limited and Data-Free Casescited by 95
- 2020Rethinking the Trigger of Backdoor Attackcited by 77
- 2018Backdooring Convolutional Neural Networks via Targeted Weight Perturbationscited by 69
- 2020Backdoor Attack against Speaker Verificationcited by 66
- 2019Model Agnostic Defence against Backdoor Attacks in Machine Learningcited by 52
- 2019NeuronInspect: Detecting Backdoors in Neural Networks via Output Explanationscited by 50
- 2020Towards Probabilistic Verification of Machine Unlearningcited by 50
- 2020Backdoor Attacks against Transfer Learning with Pre-trained Deep Learning Modelscited by 44
- 2019TrojDRL: Trojan Attacks on Deep Reinforcement Learning Agentscited by 30
- 2020ConFoc: Content-Focus Protection Against Trojan Attacks on Neural Networkscited by 28
- 2020Adversarial Profiles: Detecting Out-Distribution & Adversarial Samples in Pre-trained CNNscited by 1
Built on by
Citing papers per year: 2022: 2; 2023: 1.
3 papers.
- 2023Gradient Shaping: Enhancing Backdoor Attack Against Reverse Engineering29 Jan 2023
- 2022Circumventing interpretability: How to defeat mind-readers21 Dec 2022
- 2022Game of Trojans: A Submodular Byzantine Approach13 Jul 2022
Similar, not cited
Close in wording, no citation either way
- 2022Defense Against Multi-target Trojan Attacks
- 2021Test-Time Detection of Backdoor Triggers for Poisoned Deep Neural Networks
- 2023Backdoor Mitigation by Correcting the Distribution of Neural Activations
- 2021CatchBackdoor: Backdoor Detection via Critical Trojan Neural Path Fuzzing
- 2022M-to-N Backdoor Paradigm: A Multi-Trigger and Multi-Target Attack to Deep Learning Models
- 2022Architectural Backdoors in Neural Networks
- 2021SGBA: A Stealthy Scapegoat Backdoor Attack against Deep Neural Networks
- 2022Imperceptible Backdoor Attack: From Input Space to Feature Representation