# Model-Reuse Attacks on Deep Learning Systems

arXiv: https://arxiv.org/abs/1812.00483 · HTML: https://mlsanity.com/paper/1812.00483/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Dec 2018.

## Built on by (most cited first, 30 of 49)

- [BadEncoder: Backdoor Attacks to Pre-trained Encoders in Self-Supervised Learning](https://mlsanity.com/paper/2108.00352/) (2021)
- [Software Engineering for AI-Based Systems: A Survey](https://mlsanity.com/paper/2105.01984/) (2021)
- [T-Miner: A Generative Approach to Defend Against Trojan Attacks on DNN-based Text Classification](https://mlsanity.com/paper/2103.04264/) (2021)
- [Relating Adversarially Robust Generalization to Flat Minima](https://mlsanity.com/paper/2104.04448/) (2021)
- [AEVA: Black-box Backdoor Detection Using Adversarial Extreme Value Analysis](https://mlsanity.com/paper/2110.14880/) (2021)
- [Red Alarm for Pre-trained Models: Universal Vulnerability to Neuron-Level Backdoor Attacks](https://mlsanity.com/paper/2101.06969/) (2021)
- [Backdoor Attacks and Countermeasures in Natural Language Processing Models: A Comprehensive Security Review](https://mlsanity.com/paper/2309.06055/) (2023)
- [Pre-trained Trojan Attacks for Visual Recognition](https://mlsanity.com/paper/2312.15172/) (2023)
- [An Embarrassingly Simple Backdoor Attack on Self-supervised Learning](https://mlsanity.com/paper/2210.07346/) (2022)
- [Quantization Backdoors to Deep Learning Commercial Frameworks](https://mlsanity.com/paper/2108.09187/) (2021)
- [RoFL: Robustness of Secure Federated Learning](https://mlsanity.com/paper/2107.03311/) (2021)
- [PolicyCleanse: Backdoor Detection and Mitigation in Reinforcement Learning](https://mlsanity.com/paper/2202.03609/) (2022)
- [Reversible Watermarking in Deep Convolutional Neural Networks for Integrity Authentication](https://mlsanity.com/paper/2104.04268/) (2021)
- [Decentralized Deep Learning for Multi-Access Edge Computing: A Survey on Communication Efficiency and Trustworthiness](https://mlsanity.com/paper/2108.03980/) (2021)
- [Low-Rank Softmax Can Have Unargmaxable Classes in Theory but Rarely in Practice](https://mlsanity.com/paper/2203.06462/) (2022)
- [Versatile Weight Attack via Flipping Limited Bits](https://mlsanity.com/paper/2207.12405/) (2022)
- [CLIBE: Detecting Dynamic Backdoors in Transformer-based NLP Models](https://mlsanity.com/paper/2409.01193/) (2024)
- [Smart App Attack: Hacking Deep Learning Models in Android Apps](https://mlsanity.com/paper/2204.11075/) (2022)
- [Semi-Targeted Model Poisoning Attack on Federated Learning via Backward Error Analysis](https://mlsanity.com/paper/2203.11633/) (2022)
- [On the Difficulty of Defending Contrastive Learning against Backdoor Attacks](https://mlsanity.com/paper/2312.09057/) (2023)
- [Watch Out! Simple Horizontal Class Backdoor Can Trivially Evade Defense](https://mlsanity.com/paper/2310.00542/) (2023)
- [On the Security Risks of AutoML](https://mlsanity.com/paper/2110.06018/) (2021)
- [Measuring Discrimination to Boost Comparative Testing for Multiple Deep Learning Models](https://mlsanity.com/paper/2103.04333/) (2021)
- [Random and Adversarial Bit Error Robustness: Energy-Efficient and Secure DNN Accelerators](https://mlsanity.com/paper/2104.08323/) (2021)
- [Turning Federated Learning Systems Into Covert Channels](https://mlsanity.com/paper/2104.10561/) (2021)
- [A Synergetic Attack against Neural Network Classifiers combining Backdoor and Adversarial Examples](https://mlsanity.com/paper/2109.01275/) (2021)
- [Reusing Deep Neural Network Models through Model Re-engineering](https://mlsanity.com/paper/2304.00245/) (2023)
- [CASSOCK: Viable Backdoor Attacks against DNN in The Wall of Source-Specific Backdoor Defences](https://mlsanity.com/paper/2206.00145/) (2022)
- [An Adaptive Black-box Defense against Trojan Attacks (TrojDef)](https://mlsanity.com/paper/2209.01721/) (2022)
- [On the Security Risks of Knowledge Graph Reasoning](https://mlsanity.com/paper/2305.02383/) (2023)

## Abstract (arXiv)

Many of today's machine learning (ML) systems are built by reusing an array of, often pre-trained, primitive models, each fulfilling distinct functionality (e.g., feature extraction). The increasing use of primitive models significantly simplifies and expedites the development cycles of ML systems. Yet, because most of such models are contributed and maintained by untrusted sources, their lack of standardization or regulation entails profound security implications, about which little is known thus far. In this paper, we demonstrate that malicious primitive models pose immense threats to the security of ML systems. We present a broad class of em model-reuse attacks wherein maliciously crafted models trigger host ML systems to misbehave on targeted inputs in a highly predictable manner. By empirically studying four deep learning systems (including both individual and ensemble systems) used in skin cancer screening, speech recognition, face verification, and autonomous steering, we show that such attacks are (i) effective - the host systems misbehave on the targeted inputs as desired by the adversary with high probability, (ii) evasive - the malicious models function indistinguishably from their benign counterparts on non-targeted inputs, (iii) elastic - the malicious models remain effective regardless of various system design choices and tuning strategies, and (iv) easy - the adversary needs little prior knowledge about the data used for system tuning or inference. We provide analytical justification for the effectiveness of model-reuse attacks, which points to the unprecedented complexity of today's primitive models. This issue thus seems fundamental to many ML systems. We further discuss potential countermeasures and their challenges, which lead to several promising research directions.