# Can Adversarial Weight Perturbations Inject Neural Backdoors?

arXiv: https://arxiv.org/abs/2008.01761 · HTML: https://mlsanity.com/paper/2008.01761/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Aug 2020.

## Built on by (most cited first, 30 of 56)

- [Adversarial Unlearning of Backdoors via Implicit Hypergradient](https://mlsanity.com/paper/2110.03735/) (2021)
- [Be Careful about Poisoned Word Embeddings: Exploring the Vulnerability of the Embedding Layers in NLP Models](https://mlsanity.com/paper/2103.15543/) (2021)
- [RAP: Robustness-Aware Perturbations for Defending against Backdoor Attacks on NLP Models](https://mlsanity.com/paper/2110.07831/) (2021)
- [BadEdit: Backdooring large language models by model editing](https://mlsanity.com/paper/2403.13355/) (2024)
- [BadPre: Task-agnostic Backdoor Attacks to Pre-trained NLP Foundation Models](https://mlsanity.com/paper/2110.02467/) (2021)
- [Handcrafted Backdoors in Deep Neural Networks](https://mlsanity.com/paper/2106.04690/) (2021)
- [Reconstructive Neuron Pruning for Backdoor Defense](https://mlsanity.com/paper/2305.14876/) (2023)
- [A Study of the Attention Abnormality in Trojaned BERTs](https://mlsanity.com/paper/2205.08305/) (2022)
- [BERT & Family Eat Word Salad: Experiments with Text Understanding](https://mlsanity.com/paper/2101.03453/) (2021)
- [Test-Time Backdoor Attacks on Multimodal Large Language Models](https://mlsanity.com/paper/2402.08577/) (2024)
- [Backdoor Attacks and Countermeasures in Natural Language Processing Models: A Comprehensive Security Review](https://mlsanity.com/paper/2309.06055/) (2023)
- [A Survey of Recent Backdoor Attacks and Defenses in Large Language Models](https://mlsanity.com/paper/2406.06852/) (2024)
- [Defending Against Backdoor Attacks in Natural Language Generation](https://mlsanity.com/paper/2106.01810/) (2021)
- [Quantization Backdoors to Deep Learning Commercial Frameworks](https://mlsanity.com/paper/2108.09187/) (2021)
- [Adversarial Robustness of Neural-Statistical Features in Detection of Generative Transformers](https://mlsanity.com/paper/2203.07983/) (2022)
- [How to Inject Backdoors with Better Consistency: Logit Anchoring on Clean Data](https://mlsanity.com/paper/2109.01300/) (2021)
- [Backdoored Retrievers for Prompt Injection Attacks on Retrieval Augmented Generation of Large Language Models](https://mlsanity.com/paper/2410.14479/) (2024)
- [Robust Backdoor Attacks against Deep Neural Networks in Real Physical World](https://mlsanity.com/paper/2104.07395/) (2021)
- [Distilling Cognitive Backdoor Patterns within an Image](https://mlsanity.com/paper/2301.10908/) (2023)
- [Attacks in Adversarial Machine Learning: A Systematic Survey from the Life-cycle Perspective](https://mlsanity.com/paper/2302.09457/) (2023)
- [Qu-ANTI-zation: Exploiting Quantization Artifacts for Achieving Adversarial Outcomes](https://mlsanity.com/paper/2110.13541/) (2021)
- [ParaFuzz: An Interpretability-Driven Technique for Detecting Poisoned Samples in NLP](https://mlsanity.com/paper/2308.02122/) (2023)
- [Data and Model Poisoning Backdoor Attacks on Wireless Federated Learning, and the Defense Mechanisms: A Comprehensive Survey](https://mlsanity.com/paper/2312.08667/) (2023)
- [GA-SAM: Gradient-Strength based Adaptive Sharpness-Aware Minimization for Improved Generalization](https://mlsanity.com/paper/2210.06895/) (2022)
- [Backdoor Secrets Unveiled: Identifying Backdoor Data with Optimized Scaled Prediction Consistency](https://mlsanity.com/paper/2403.10717/) (2024)
- [Persistent Backdoor Attacks in Continual Learning](https://mlsanity.com/paper/2409.13864/) (2024)
- [Trap and Replace: Defending Backdoor Attacks by Trapping Them into an Easy-to-Replace Subnetwork](https://mlsanity.com/paper/2210.06428/) (2022)
- [Towards A Critical Evaluation of Robustness for Deep Learning Backdoor Countermeasures](https://mlsanity.com/paper/2204.06273/) (2022)
- [Selective Amnesia: On Efficient, High-Fidelity and Blind Suppression of Backdoor Effects in Trojaned Machine Learning Models](https://mlsanity.com/paper/2212.04687/) (2022)
- [Accumulative Poisoning Attacks on Real-time Data](https://mlsanity.com/paper/2106.09993/) (2021)

## Abstract (arXiv)

Adversarial machine learning has exposed several security hazards of neural models and has become an important research topic in recent times. Thus far, the concept of an "adversarial perturbation" has exclusively been used with reference to the input space referring to a small, imperceptible change which can cause a ML model to err. In this work we extend the idea of "adversarial perturbations" to the space of model weights, specifically to inject backdoors in trained DNNs, which exposes a security risk of using publicly available trained models. Here, injecting a backdoor refers to obtaining a desired outcome from the model when a trigger pattern is added to the input, while retaining the original model predictions on a non-triggered input. From the perspective of an adversary, we characterize these adversarial perturbations to be constrained within an ℓ∞ norm around the original model weights. We introduce adversarial perturbations in the model weights using a composite loss on the predictions of the original model and the desired trigger through projected gradient descent. We empirically show that these adversarial weight perturbations exist universally across several computer vision and natural language processing tasks. Our results show that backdoors can be successfully injected with a very small average relative change in model weight values for several applications.