# ImpNet: Imperceptible and blackbox-undetectable backdoors in compiled neural networks

arXiv: https://arxiv.org/abs/2210.00108 · HTML: https://mlsanity.com/paper/2210.00108/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 70 references that are in the map.
- First posted on arXiv 30 Sep 2022.

## Builds on (most cited first, 30 of 39)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017)
- [Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning](https://mlsanity.com/paper/1712.05526/) (2017)
- [Practical Black-Box Attacks against Machine Learning](https://mlsanity.com/paper/1602.02697/) (2016)
- [The Limitations of Deep Learning in Adversarial Settings](https://mlsanity.com/paper/1511.07528/) (2015)
- [Evasion Attacks against Machine Learning at Test Time](https://mlsanity.com/paper/1708.06131/) (2017)
- [Backdoor Learning: A Survey](https://mlsanity.com/paper/2007.08745/) (2020)
- [Adversarial Examples: Attacks and Defenses for Deep Learning](https://mlsanity.com/paper/1712.07107/) (2017)
- [Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks](https://mlsanity.com/paper/2007.02343/) (2020)
- [Caffe: Convolutional Architecture for Fast Feature Embedding](https://mlsanity.com/paper/1408.5093/) (2014)
- [Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger](https://mlsanity.com/paper/2105.12400/) (2021)
- [BadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvements](https://mlsanity.com/paper/2006.01043/) (2020)
- [Blind Backdoors in Deep Learning Models](https://mlsanity.com/paper/2005.03823/) (2020)
- [On the Necessity of Auditable Algorithmic Definitions for Machine Unlearning](https://mlsanity.com/paper/2110.11891/) (2021)
- [Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer](https://mlsanity.com/paper/2110.07139/) (2021)
- [Entangled Watermarks as a Defense against Model Extraction](https://mlsanity.com/paper/2002.12200/) (2020)
- [Sponge Examples: Energy-Latency Attacks on Neural Networks](https://mlsanity.com/paper/2006.03463/) (2020)
- [An Embarrassingly Simple Approach for Trojan Attack in Deep Neural Networks](https://mlsanity.com/paper/2006.08131/) (2020)
- [Deep Feature Space Trojan Attack of Neural Networks by Controlled Detoxification](https://mlsanity.com/paper/2012.11212/) (2020)
- [Turn the Combination Lock: Learnable Textual Backdoor Attacks via Word Substitution](https://mlsanity.com/paper/2106.06361/) (2021)
- [Exploring Connections Between Active Learning and Model Extraction](https://mlsanity.com/paper/1811.02054/) (2018)
- [Proof-of-Learning: Definitions and Practice](https://mlsanity.com/paper/2103.05633/) (2021)
- [On Certifying Robustness against Backdoor Attacks via Randomized Smoothing](https://mlsanity.com/paper/2002.11750/) (2020)
- [Manipulating SGD with Data Ordering Attacks](https://mlsanity.com/paper/2104.09667/) (2021)
- [Bad Characters: Imperceptible NLP Attacks](https://mlsanity.com/paper/2106.09898/) (2021)
- [Handcrafted Backdoors in Deep Neural Networks](https://mlsanity.com/paper/2106.04690/) (2021)
- [Planting Undetectable Backdoors in Machine Learning Models](https://mlsanity.com/paper/2204.06974/) (2022)
- [DeepPayload: Black-box Backdoor Attack on Deep Learning Models through Neural Payload Injection](https://mlsanity.com/paper/2101.06896/) (2021)

## Built on by (most cited first, 9 of 9)

- [Architectural Neural Backdoors from First Principles](https://mlsanity.com/paper/2402.06957/) (2024)
- [Your Compiler is Backdooring Your Model: Understanding and Exploiting Compilation Inconsistency Vulnerabilities in Deep Learning Compilers](https://mlsanity.com/paper/2509.11173/) (2025)
- [Machine Learning needs Better Randomness Standards: Randomised Smoothing and PRNG-based attacks](https://mlsanity.com/paper/2306.14043/) (2023)
- [Performance-lossless Black-box Model Watermarking](https://mlsanity.com/paper/2312.06488/) (2023)
- [Understanding CNN Hidden Neuron Activations Using Structured Background Knowledge and Deductive Reasoning](https://mlsanity.com/paper/2308.03999/) (2023)
- [Everyone Can Attack: Repurpose Lossy Compression as a Natural Backdoor Attack](https://mlsanity.com/paper/2308.16684/) (2023)
- [Backdoor Detection through Replicated Execution of Outsourced Training](https://mlsanity.com/paper/2504.00170/) (2025)
- [Trusted Weights, Treacherous Optimizations? Optimization-Triggered Backdoor Attacks on LLMs](https://mlsanity.com/paper/2605.20641/) (2026)
- [Evil from Within: Machine Learning Backdoors through Hardware Trojans](https://mlsanity.com/paper/2304.08411/) (2023)

## Abstract (arXiv)

Early backdoor attacks against machine learning set off an arms race in attack and defence development. Defences have since appeared demonstrating some ability to detect backdoors in models or even remove them. These defences work by inspecting the training data, the model, or the integrity of the training procedure. In this work, we show that backdoors can be added during compilation, circumventing any safeguards in the data preparation and model training stages. The attacker can not only insert existing weight-based backdoors during compilation, but also a new class of weight-independent backdoors, such as ImpNet. These backdoors are impossible to detect during the training or data preparation processes, because they are not yet present. Next, we demonstrate that some backdoors, including ImpNet, can only be reliably detected at the stage where they are inserted and removing them anywhere else presents a significant challenge. We conclude that ML model security requires assurance of provenance along the entire technical pipeline, including the data, model architecture, compiler, and hardware specification.