# An Embarrassingly Simple Approach for Trojan Attack in Deep Neural Networks

arXiv: https://arxiv.org/abs/2006.08131 · HTML: https://mlsanity.com/paper/2006.08131/

## Key facts

- Cited by 97 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jun 2020.

## Built on by (most cited first, 30 of 97)

- [Backdoor Defense via Decoupling the Training Process](https://mlsanity.com/paper/2202.03423/) (2022)
- [Shortcut Learning of Large Language Models in Natural Language Understanding](https://mlsanity.com/paper/2208.11857/) (2022)
- [Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU Models](https://mlsanity.com/paper/2103.06922/) (2021)
- [Black-box Dataset Ownership Verification via Backdoor Watermarking](https://mlsanity.com/paper/2209.06015/) (2022)
- [Handcrafted Backdoors in Deep Neural Networks](https://mlsanity.com/paper/2106.04690/) (2021)
- [Availability Attacks Create Shortcuts](https://mlsanity.com/paper/2111.00898/) (2021)
- [Black-box Backdoor Defense via Zero-shot Image Purification](https://mlsanity.com/paper/2303.12175/) (2023)
- [Backdoor Attacks Against Deep Image Compression via Adaptive Frequency Trigger](https://mlsanity.com/paper/2302.14677/) (2023)
- [Test-Time Backdoor Attacks on Multimodal Large Language Models](https://mlsanity.com/paper/2402.08577/) (2024)
- [Training with More Confidence: Mitigating Injected and Natural Backdoors During Training](https://mlsanity.com/paper/2202.06382/) (2022)
- [Did You Train on My Dataset? Towards Public Dataset Protection with Clean-Label Backdoor Watermarking](https://mlsanity.com/paper/2303.11470/) (2023)
- [PointBA: Towards Backdoor Attacks in 3D Point Cloud](https://mlsanity.com/paper/2103.16074/) (2021)
- [Towards Practical Deployment-Stage Backdoor Attack on Deep Neural Networks](https://mlsanity.com/paper/2111.12965/) (2021)
- [Label Poisoning is All You Need](https://mlsanity.com/paper/2310.18933/) (2023)
- [BackdoorBox: A Python Toolbox for Backdoor Learning](https://mlsanity.com/paper/2302.01762/) (2023)
- [Rethinking Graph Backdoor Attacks: A Distribution-Preserving Perspective](https://mlsanity.com/paper/2405.10757/) (2024)
- [Attacks in Adversarial Machine Learning: A Systematic Survey from the Life-cycle Perspective](https://mlsanity.com/paper/2302.09457/) (2023)
- [When Backdoors Speak: Understanding LLM Backdoor Attacks Through Model-Generated Explanations](https://mlsanity.com/paper/2411.12701/) (2024)
- [Setting the Trap: Capturing and Defeating Backdoors in Pretrained Language Models through Honeypots](https://mlsanity.com/paper/2310.18633/) (2023)
- [Stealthy Backdoors as Compression Artifacts](https://mlsanity.com/paper/2104.15129/) (2021)
- [Backdoor Attack through Frequency Domain](https://mlsanity.com/paper/2111.10991/) (2021)
- [Backdoor Federated Learning by Poisoning Backdoor-Critical Layers](https://mlsanity.com/paper/2308.04466/) (2023)
- [A Survey of Neural Trojan Attacks and Defenses in Deep Learning](https://mlsanity.com/paper/2202.07183/) (2022)
- [AI Risk Management Should Incorporate Both Safety and Security](https://mlsanity.com/paper/2405.19524/) (2024)
- [Enhancing Backdoor Attacks with Multi-Level MMD Regularization](https://mlsanity.com/paper/2111.05077/) (2021)
- [Get a Model! Model Hijacking Attack Against Machine Learning Models](https://mlsanity.com/paper/2111.04394/) (2021)
- [Fall of Giants: How popular text-based MLaaS fall against a simple evasion attack](https://mlsanity.com/paper/2104.05996/) (2021)
- [Backdoor Secrets Unveiled: Identifying Backdoor Data with Optimized Scaled Prediction Consistency](https://mlsanity.com/paper/2403.10717/) (2024)
- [Selective Amnesia: On Efficient, High-Fidelity and Blind Suppression of Backdoor Effects in Trojaned Machine Learning Models](https://mlsanity.com/paper/2212.04687/) (2022)
- [Backdoor Attacks on Time Series: A Generative Approach](https://mlsanity.com/paper/2211.07915/) (2022)

## Abstract (arXiv)

With the widespread use of deep neural networks (DNNs) in high-stake applications, the security problem of the DNN models has received extensive attention. In this paper, we investigate a specific security problem called trojan attack, which aims to attack deployed DNN systems relying on the hidden trigger patterns inserted by malicious hackers. We propose a training-free attack approach which is different from previous work, in which trojaned behaviors are injected by retraining model on a poisoned dataset. Specifically, we do not change parameters in the original model but insert a tiny trojan module (TrojanNet) into the target model. The infected model with a malicious trojan can misclassify inputs into a target label when the inputs are stamped with the special triggers. The proposed TrojanNet has several nice properties including (1) it activates by tiny trigger patterns and keeps silent for other signals, (2) it is model-agnostic and could be injected into most DNNs, dramatically expanding its attack scenarios, and (3) the training-free mechanism saves massive training efforts comparing to conventional trojan attack methods. The experimental results show that TrojanNet can inject the trojan into all labels simultaneously (all-label trojan attack) and achieves 100% attack success rate without affecting model accuracy on original tasks. Experimental analysis further demonstrates that state-of-the-art trojan detection algorithms fail to detect TrojanNet attack. The code is available at https://github.com/trx14/TrojanNet.