# Defense against Backdoor Attacks via Identifying and Purifying Bad Neurons

arXiv: https://arxiv.org/abs/2208.06537 · HTML: https://mlsanity.com/paper/2208.06537/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 25 references that are in the map.
- First posted on arXiv 13 Aug 2022.

## Builds on (most cited first, 18 of 18)

- [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017)
- [Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning](https://mlsanity.com/paper/1712.05526/) (2017)
- [Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks](https://mlsanity.com/paper/1805.12185/) (2018)
- [Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networks](https://mlsanity.com/paper/1804.00792/) (2018)
- [STRIP: A Defence Against Trojan Attacks on Deep Neural Networks](https://mlsanity.com/paper/1902.06531/) (2019)
- [Invisible Backdoor Attack with Sample-Specific Triggers](https://mlsanity.com/paper/2012.03816/) (2020)
- [Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks](https://mlsanity.com/paper/2101.05930/) (2021)
- [A new Backdoor Attack in CNNs by training set corruption without label poisoning](https://mlsanity.com/paper/1902.11237/) (2019)
- [Detecting AI Trojans Using Meta Neural Analysis](https://mlsanity.com/paper/1910.03137/) (2019)
- [Adversarial Unlearning of Backdoors via Implicit Hypergradient](https://mlsanity.com/paper/2110.03735/) (2021)
- [Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbation](https://mlsanity.com/paper/1808.10307/) (2018)
- [Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustness](https://mlsanity.com/paper/2005.00060/) (2020)
- [Neuron Shapley: Discovering the Responsible Neurons](https://mlsanity.com/paper/2002.09815/) (2020)
- [Practical Detection of Trojan Neural Networks: Data-Limited and Data-Free Cases](https://mlsanity.com/paper/2007.15802/) (2020)
- [Rethinking the Trigger of Backdoor Attack](https://mlsanity.com/paper/2004.04692/) (2020)
- [Optimization for deep learning: theory and algorithms](https://mlsanity.com/paper/1912.08957/) (2019)
- [How to Inject Backdoors with Better Consistency: Logit Anchoring on Clean Data](https://mlsanity.com/paper/2109.01300/) (2021)
- [Systematic Evaluation of Backdoor Data Poisoning Attacks on Image Classifiers](https://mlsanity.com/paper/2004.11514/) (2020)

## Built on by (most cited first, 4 of 4)

- [On the Trustworthiness Landscape of State-of-the-art Generative Models: A Survey and Outlook](https://mlsanity.com/paper/2307.16680/) (2023)
- [On the Robustness of Split Learning against Adversarial Attacks](https://mlsanity.com/paper/2307.07916/) (2023)
- [PBP: Post-training Backdoor Purification for Malware Classifiers](https://mlsanity.com/paper/2412.03441/) (2024)
- [SATBA: An Invisible Backdoor Attack Based On Spatial Attention](https://mlsanity.com/paper/2302.13056/) (2023)

## Abstract (arXiv)

The opacity of neural networks leads their vulnerability to backdoor attacks, where hidden attention of infected neurons is triggered to override normal predictions to the attacker-chosen ones. In this paper, we propose a novel backdoor defense method to mark and purify the infected neurons in the backdoored neural networks. Specifically, we first define a new metric, called benign salience. By combining the first-order gradient to retain the connections between neurons, benign salience can identify the infected neurons with higher accuracy than the commonly used metric in backdoor defense. Then, a new Adaptive Regularization (AR) mechanism is proposed to assist in purifying these identified infected neurons via fine-tuning. Due to the ability to adapt to different magnitudes of parameters, AR can provide faster and more stable convergence than the common regularization mechanism in neuron purifying. Extensive experimental results demonstrate that our method can erase the backdoor in neural networks with negligible performance degradation.