# Detecting Backdoors in Neural Networks Using Novel Feature-Based Anomaly Detection

arXiv: https://arxiv.org/abs/2011.02526 · HTML: https://mlsanity.com/paper/2011.02526/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Nov 2020.

## Built on by (most cited first, 9 of 9)

- [A Survey of Neural Trojan Attacks and Defenses in Deep Learning](https://mlsanity.com/paper/2202.07183/) (2022)
- [Jigsaw Puzzle: Selective Backdoor Attack to Subvert Malware Classifiers](https://mlsanity.com/paper/2202.05470/) (2022)
- [Preference Poisoning Attacks on Reward Model Learning](https://mlsanity.com/paper/2402.01920/) (2024)
- [Differential Analysis of Triggers and Benign Features for Black-Box DNN Backdoor Detection](https://mlsanity.com/paper/2307.05422/) (2023)
- [EX-RAY: Distinguishing Injected Backdoor from Natural Features in Neural Networks by Examining Differential Feature Symmetry](https://mlsanity.com/paper/2103.08820/) (2021)
- [Backdoor Learning for NLP: Recent Advances, Challenges, and Future Research Directions](https://mlsanity.com/paper/2302.06801/) (2023)
- [Towards Robust Vision Transformer via Masked Adaptive Ensemble](https://mlsanity.com/paper/2407.15385/) (2024)
- [UCM-Net: A Lightweight and Efficient Solution for Skin Lesion Segmentation using MLP and CNN](https://mlsanity.com/paper/2310.09457/) (2023)
- [An Upper Bound for the Distribution Overlap Index and Its Applications](https://mlsanity.com/paper/2212.08701/) (2022)

## Abstract (arXiv)

This paper proposes a new defense against neural network backdooring attacks that are maliciously trained to mispredict in the presence of attacker-chosen triggers. Our defense is based on the intuition that the feature extraction layers of a backdoored network embed new features to detect the presence of a trigger and the subsequent classification layers learn to mispredict when triggers are detected. Therefore, to detect backdoors, the proposed defense uses two synergistic anomaly detectors trained on clean validation data: the first is a novelty detector that checks for anomalous features, while the second detects anomalous mappings from features to outputs by comparing with a separate classifier trained on validation data. The approach is evaluated on a wide range of backdoored networks (with multiple variations of triggers) that successfully evade state-of-the-art defenses. Additionally, we evaluate the robustness of our approach on imperceptible perturbations, scalability on large-scale datasets, and effectiveness under domain shift. This paper also shows that the defense can be further improved using data augmentation.