# TOP: Backdoor Detection in Neural Networks via Transferability of Perturbation

arXiv: https://arxiv.org/abs/2103.10274 · HTML: https://mlsanity.com/paper/2103.10274/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 40 references that are in the map.
- First posted on arXiv 18 Mar 2021.

## Builds on (most cited first, 24 of 24)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017)
- [Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning](https://mlsanity.com/paper/1712.05526/) (2017)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [Universal adversarial perturbations](https://mlsanity.com/paper/1610.08401/) (2016)
- [Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks](https://mlsanity.com/paper/1805.12185/) (2018)
- [STRIP: A Defence Against Trojan Attacks on Deep Neural Networks](https://mlsanity.com/paper/1902.06531/) (2019)
- [Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering](https://mlsanity.com/paper/1811.03728/) (2018)
- [Backdoor Learning: A Survey](https://mlsanity.com/paper/2007.08745/) (2020)
- [Hidden Trigger Backdoor Attacks](https://mlsanity.com/paper/1910.00033/) (2019)
- [Neural Trojans](https://mlsanity.com/paper/1710.00942/) (2017)
- [SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systems](https://mlsanity.com/paper/1812.00292/) (2018)
- [Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review](https://mlsanity.com/paper/2007.10760/) (2020)
- [TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systems](https://mlsanity.com/paper/1908.01763/) (2019)
- [Backdoor Scanning for Deep Neural Networks through K-Arm Optimization](https://mlsanity.com/paper/2102.05123/) (2021)
- [Practical Detection of Trojan Neural Networks: Data-Limited and Data-Free Cases](https://mlsanity.com/paper/2007.15802/) (2020)
- [DeepSweep: An Evaluation Framework for Mitigating DNN Backdoor Attacks using Data Augmentation](https://mlsanity.com/paper/2012.07006/) (2020)
- [advertorch v0.1: An Adversarial Robustness Toolbox based on PyTorch](https://mlsanity.com/paper/1902.07623/) (2019)
- [The TrojAI Software Framework: An OpenSource tool for Embedding Trojans into Deep Learning Models](https://mlsanity.com/paper/2003.07233/) (2020)
- [Poisoned classifiers are not only backdoored, they are fundamentally broken](https://mlsanity.com/paper/2010.09080/) (2020)
- [Baseline Pruning-Based Approach to Trojan Detection in Neural Networks](https://mlsanity.com/paper/2101.12016/) (2021)

## Built on by (most cited first, 13 of 13)

- [Trigger Hunting with a Topological Prior for Trojan Detection](https://mlsanity.com/paper/2110.08335/) (2021)
- [UMD: Unsupervised Model Detection for X2X Backdoor Attacks](https://mlsanity.com/paper/2305.18651/) (2023)
- [A Survey of Neural Trojan Attacks and Defenses in Deep Learning](https://mlsanity.com/paper/2202.07183/) (2022)
- [Preference Poisoning Attacks on Reward Model Learning](https://mlsanity.com/paper/2402.01920/) (2024)
- [Selective Amnesia: On Efficient, High-Fidelity and Blind Suppression of Backdoor Effects in Trojaned Machine Learning Models](https://mlsanity.com/paper/2212.04687/) (2022)
- [Accumulative Poisoning Attacks on Real-time Data](https://mlsanity.com/paper/2106.09993/) (2021)
- [Single Image Backdoor Inversion via Robust Smoothed Classifiers](https://mlsanity.com/paper/2303.00215/) (2023)
- [Check Your Other Door! Creating Backdoor Attacks in the Frequency Domain](https://mlsanity.com/paper/2109.05507/) (2021)
- [Don't FREAK Out: A Frequency-Inspired Approach to Detecting Backdoor Poisoned Samples in DNNs](https://mlsanity.com/paper/2303.13211/) (2023)
- [MISA: Online Defense of Trojaned Models using Misattributions](https://mlsanity.com/paper/2103.15918/) (2021)
- [An Adaptive Black-box Backdoor Detection Method for Deep Neural Networks](https://mlsanity.com/paper/2204.04329/) (2022)
- [Solving Trojan Detection Competitions with Linear Weight Classification](https://mlsanity.com/paper/2411.03445/) (2024)
- [Dormant Neural Trojans](https://mlsanity.com/paper/2211.01808/) (2022)

## Abstract (arXiv)

Deep neural networks (DNNs) are vulnerable to "backdoor" poisoning attacks, in which an adversary implants a secret trigger into an otherwise normally functioning model. Detection of backdoors in trained models without access to the training data or example triggers is an important open problem. In this paper, we identify an interesting property of these models: adversarial perturbations transfer from image to image more readily in poisoned models than in clean models. This holds for a variety of model and trigger types, including triggers that are not linearly separable from clean data. We use this feature to detect poisoned models in the TrojAI benchmark, as well as additional models.