# Handcrafted Backdoors in Deep Neural Networks

arXiv: https://arxiv.org/abs/2106.04690 · HTML: https://mlsanity.com/paper/2106.04690/

## Key facts

- Cited by 63 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 69 references that are in the map.
- First posted on arXiv 8 Jun 2021.

## Builds on (most cited first, 30 of 40)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [Certified Adversarial Robustness via Randomized Smoothing](https://mlsanity.com/paper/1902.02918/) (2019)
- [Pruning Filters for Efficient ConvNets](https://mlsanity.com/paper/1608.08710/) (2016)
- [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017)
- [Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning](https://mlsanity.com/paper/1712.05526/) (2017)
- [Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks](https://mlsanity.com/paper/1702.01135/) (2017)
- [Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks](https://mlsanity.com/paper/1805.12185/) (2018)
- [Spectral Signatures in Backdoor Attacks](https://mlsanity.com/paper/1811.00636/) (2018)
- [Certified Robustness to Adversarial Examples with Differential Privacy](https://mlsanity.com/paper/1802.03471/) (2018)
- [STRIP: A Defence Against Trojan Attacks on Deep Neural Networks](https://mlsanity.com/paper/1902.06531/) (2019)
- [Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering](https://mlsanity.com/paper/1811.03728/) (2018)
- [Deep Speech 2: End-to-End Speech Recognition in English and Mandarin](https://mlsanity.com/paper/1512.02595/) (2015)
- [Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability](https://mlsanity.com/paper/2103.00065/) (2021)
- [Hidden Trigger Backdoor Attacks](https://mlsanity.com/paper/1910.00033/) (2019)
- [Deep Speech: Scaling up end-to-end speech recognition](https://mlsanity.com/paper/1412.5567/) (2014)
- [Certified Defenses for Data Poisoning Attacks](https://mlsanity.com/paper/1706.03691/) (2017)
- [Label-Consistent Backdoor Attacks](https://mlsanity.com/paper/1912.02771/) (2019)
- [Detecting AI Trojans Using Meta Neural Analysis](https://mlsanity.com/paper/1910.03137/) (2019)
- [Blind Backdoors in Deep Learning Models](https://mlsanity.com/paper/2005.03823/) (2020)
- [On the Turing Completeness of Modern Neural Network Architectures](https://mlsanity.com/paper/1901.03429/) (2019)
- [Demon in the Variant: Statistical Analysis of DNNs for Robust Backdoor Contamination Detection](https://mlsanity.com/paper/1908.00686/) (2019)
- [TBT: Targeted Neural Network Attack with Bit Trojan](https://mlsanity.com/paper/1909.05193/) (2019)
- [Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware](https://mlsanity.com/paper/1806.03287/) (2018)
- [On the importance of single directions for generalization](https://mlsanity.com/paper/1803.06959/) (2018)

## Built on by (most cited first, 30 of 63)

- [Unsolved Problems in ML Safety](https://mlsanity.com/paper/2109.13916/) (2021)
- [FLIRT: Feedback Loop In-context Red Teaming](https://mlsanity.com/paper/2308.04265/) (2023)
- [BadChain: Backdoor Chain-of-Thought Prompting for Large Language Models](https://mlsanity.com/paper/2401.12242/) (2024)
- [Spinning Language Models: Risks of Propaganda-As-A-Service and Countermeasures](https://mlsanity.com/paper/2112.05224/) (2021)
- [Planting Undetectable Backdoors in Machine Learning Models](https://mlsanity.com/paper/2204.06974/) (2022)
- [Black-box Backdoor Defense via Zero-shot Image Purification](https://mlsanity.com/paper/2303.12175/) (2023)
- [An Overview of Backdoor Attacks Against Deep Neural Networks and Possible Defences](https://mlsanity.com/paper/2111.08429/) (2021)
- [Poison Forensics: Traceback of Data Poisoning Attacks in Neural Networks](https://mlsanity.com/paper/2110.06904/) (2021)
- [Training-free Lexical Backdoor Attacks on Language Models](https://mlsanity.com/paper/2302.04116/) (2023)
- [Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Models](https://mlsanity.com/paper/2404.01231/) (2024)
- [Label Poisoning is All You Need](https://mlsanity.com/paper/2310.18933/) (2023)
- [Architectural Backdoors in Neural Networks](https://mlsanity.com/paper/2206.07840/) (2022)
- [Attacks in Adversarial Machine Learning: A Systematic Survey from the Life-cycle Perspective](https://mlsanity.com/paper/2302.09457/) (2023)
- [Data and Model Poisoning Backdoor Attacks on Wireless Federated Learning, and the Defense Mechanisms: A Comprehensive Survey](https://mlsanity.com/paper/2312.08667/) (2023)
- [Towards more Practical Threat Models in Artificial Intelligence Security](https://mlsanity.com/paper/2311.09994/) (2023)
- [BadToken: Token-level Backdoor Attacks to Multi-modal Large Language Models](https://mlsanity.com/paper/2503.16023/) (2025)
- [Quarantine: Sparsity Can Uncover the Trojan Attack Trigger for Free](https://mlsanity.com/paper/2205.11819/) (2022)
- [BaDExpert: Extracting Backdoor Functionality for Accurate Backdoor Input Detection](https://mlsanity.com/paper/2308.12439/) (2023)
- [Architectural Neural Backdoors from First Principles](https://mlsanity.com/paper/2402.06957/) (2024)
- [ImpNet: Imperceptible and blackbox-undetectable backdoors in compiled neural networks](https://mlsanity.com/paper/2210.00108/) (2022)
- [BAN: Detecting Backdoors Activated by Adversarial Neuron Noise](https://mlsanity.com/paper/2405.19928/) (2024)
- [Universal Backdoor Attacks](https://mlsanity.com/paper/2312.00157/) (2023)
- [Data Free Backdoor Attacks](https://mlsanity.com/paper/2412.06219/) (2024)
- [SoK: A Systematic Evaluation of Backdoor Trigger Characteristics in Image Classification](https://mlsanity.com/paper/2302.01740/) (2023)
- [TrojFM: Resource-efficient Backdoor Attacks against Very Large Foundation Models](https://mlsanity.com/paper/2405.16783/) (2024)
- [Exploiting Leaderboards for Large-Scale Distribution of Malicious Models](https://mlsanity.com/paper/2507.08983/) (2025)
- [Facial Misrecognition Systems: Simple Weight Manipulations Force DNNs to Err Only on Specific Persons](https://mlsanity.com/paper/2301.03118/) (2023)
- [Unelicitable Backdoors in Language Models via Cryptographic Transformer Circuits](https://mlsanity.com/paper/2406.02619/) (2024)
- [Graph Neural Backdoor: Fundamentals, Methodologies, Applications, and Future Directions](https://mlsanity.com/paper/2406.10573/) (2024)
- [LoRA as Oracle](https://mlsanity.com/paper/2601.11207/) (2026)

## Abstract (arXiv)

When machine learning training is outsourced to third parties, backdoor attacks become practical as the third party who trains the model may act maliciously to inject hidden behaviors into the otherwise accurate model. Until now, the mechanism to inject backdoors has been limited to poisoning. We argue that a supply-chain attacker has more attack techniques available by introducing a handcrafted attack that directly manipulates a model's weights. This direct modification gives our attacker more degrees of freedom compared to poisoning, and we show it can be used to evade many backdoor detection or removal defenses effectively. Across four datasets and four network architectures our backdoor attacks maintain an attack success rate above 96%. Our results suggest that further research is needed for understanding the complete space of supply-chain backdoor attacks.