ImpNet: Imperceptible and blackbox-undetectable backdoors in compiled neural networks

arXiv 2210.00108 ↗First posted 30 Sep 2022cs.LG cs.CRcited by 9

Not on the map. Why?

Abstract (arXiv)

Early backdoor attacks against machine learning set off an arms race in attack and defence development. Defences have since appeared demonstrating some ability to detect backdoors in models or even remove them. These defences work by inspecting the training data, the model, or the integrity of the training procedure. In this work, we show that backdoors can be added during compilation, circumventing any safeguards in the data preparation and model training stages. The attacker can not only insert existing weight-based backdoors during compilation, but also a new class of weight-independent backdoors, such as ImpNet. These backdoors are impossible to detect during the training or data preparation processes, because they are not yet present. Next, we demonstrate that some backdoors, including ImpNet, can only be reliably detected at the stage where they are inserted and removing them anywhere else presents a significant challenge. We conclude that ML model security requires assurance of provenance along the entire technical pipeline, including the data, model architecture, compiler, and hardware specification.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 39 of its 70 references that are in the map. First posted on arXiv 30 Sep 2022.

Builds on

39 of 70 references.

References · 39

2014Explaining and Harnessing Adversarial Examplescited by 7,364
2013Intriguing properties of neural networkscited by 5,155
2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,106
2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,055
2016Practical Black-Box Attacks against Machine Learningcited by 903
2015The Limitations of Deep Learning in Adversarial Settingscited by 790
2017Evasion Attacks against Machine Learning at Test Timecited by 689
2020Backdoor Learning: A Surveycited by 426
2017Adversarial Examples: Attacks and Defenses for Deep Learningcited by 351
2020Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networkscited by 349
2014Caffe: Convolutional Architecture for Fast Feature Embeddingcited by 288

27 more

2021Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Triggercited by 222
2020BadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvementscited by 211
2020Blind Backdoors in Deep Learning Modelscited by 182
2021On the Necessity of Auditable Algorithmic Definitions for Machine Unlearningcited by 171
2021Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfercited by 158
2020Entangled Watermarks as a Defense against Model Extractioncited by 149
2020Sponge Examples: Energy-Latency Attacks on Neural Networkscited by 111
2020An Embarrassingly Simple Approach for Trojan Attack in Deep Neural Networkscited by 97
2020Deep Feature Space Trojan Attack of Neural Networks by Controlled Detoxificationcited by 95
2021Turn the Combination Lock: Learnable Textual Backdoor Attacks via Word Substitutioncited by 89
2018Exploring Connections Between Active Learning and Model Extractioncited by 85
2021Proof-of-Learning: Definitions and Practicecited by 84
2020On Certifying Robustness against Backdoor Attacks via Randomized Smoothingcited by 77
2021Manipulating SGD with Data Ordering Attackscited by 74
2021Bad Characters: Imperceptible NLP Attackscited by 71
2021Handcrafted Backdoors in Deep Neural Networkscited by 63
2022Planting Undetectable Backdoors in Machine Learning Modelscited by 59
2021DeepPayload: Black-box Backdoor Attack on Deep Learning Models through Neural Payload Injectioncited by 44
2022Architectural Backdoors in Neural Networkscited by 25
2018Relay: A New IR for Machine Learning Frameworkscited by 24
2021Subnet Replacement: Deployment-stage backdoor attack against deep neural networks in gray-box settingcited by 21
2022On the Limitations of Stochastic Pre-processing Defensescited by 21
2021Self-Checking Deep Neural Networks in Deploymentcited by 18
2021Backdoor Attack through Frequency Domaincited by 18
2021SoK: Machine Learning Governancecited by 10
2020NNoculation: Catching BadNets in the Wildcited by 9
2018How a simple bug in ML compiler could be exploited for backdoors?cited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 1; 2025: 2; 2026: 1.

9 papers.

2024Architectural Neural Backdoors from First Principlescited by 11
2025Your Compiler is Backdooring Your Model: Understanding and Exploiting Compilation Inconsistency Vulnerabilities in Deep Learning Compilerscited by 7
2023Machine Learning needs Better Randomness Standards: Randomised Smoothing and PRNG-based attackscited by 6
2023Performance-lossless Black-box Model Watermarkingcited by 3
2023Understanding CNN Hidden Neuron Activations Using Structured Background Knowledge and Deductive Reasoningcited by 2
2023Everyone Can Attack: Repurpose Lossy Compression as a Natural Backdoor Attackcited by 1
2025Backdoor Detection through Replicated Execution of Outsourced Trainingcited by 1
2026Trusted Weights, Treacherous Optimizations? Optimization-Triggered Backdoor Attacks on LLMscited by 1
2023Evil from Within: Machine Learning Backdoors through Hardware Trojanscited by 0

Similar, not cited

Close in wording, no citation either way