# An Adaptive Black-box Backdoor Detection Method for Deep Neural Networks

arXiv: https://arxiv.org/abs/2204.04329 · HTML: https://mlsanity.com/paper/2204.04329/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 44 references that are in the map.
- First posted on arXiv 8 Apr 2022.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017)
- [Universal adversarial perturbations](https://mlsanity.com/paper/1610.08401/) (2016)
- [Fast is better than free: Revisiting adversarial training](https://mlsanity.com/paper/2001.03994/) (2020)
- [Invisible Backdoor Attack with Sample-Specific Triggers](https://mlsanity.com/paper/2012.03816/) (2020)
- [Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks](https://mlsanity.com/paper/2007.02343/) (2020)
- [Input-Aware Dynamic Backdoor Attack](https://mlsanity.com/paper/2010.08138/) (2020)
- [Certified Defenses for Data Poisoning Attacks](https://mlsanity.com/paper/1706.03691/) (2017)
- [Audio Adversarial Examples: Targeted Attacks on Speech-to-Text](https://mlsanity.com/paper/1801.01944/) (2018)
- [A backdoor attack against LSTM-based text classification systems](https://mlsanity.com/paper/1905.12457/) (2019)
- [Clean-Label Backdoor Attacks on Video Recognition Models](https://mlsanity.com/paper/2003.03030/) (2020)
- [TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systems](https://mlsanity.com/paper/1908.01763/) (2019)
- [Be Careful about Poisoned Word Embeddings: Exploring the Vulnerability of the Embedding Layers in NLP Models](https://mlsanity.com/paper/2103.15543/) (2021)
- [Deep Feature Space Trojan Attack of Neural Networks by Controlled Detoxification](https://mlsanity.com/paper/2012.11212/) (2020)
- [Rethinking the Trigger of Backdoor Attack](https://mlsanity.com/paper/2004.04692/) (2020)
- [DeepID3: Face Recognition with Very Deep Neural Networks](https://mlsanity.com/paper/1502.00873/) (2015)
- [AEVA: Black-box Backdoor Detection Using Adversarial Extreme Value Analysis](https://mlsanity.com/paper/2110.14880/) (2021)
- [The TrojAI Software Framework: An OpenSource tool for Embedding Trojans into Deep Learning Models](https://mlsanity.com/paper/2003.07233/) (2020)
- [Systematic Evaluation of Backdoor Data Poisoning Attacks on Image Classifiers](https://mlsanity.com/paper/2004.11514/) (2020)
- [CLEANN: Accelerated Trojan Shield for Embedded Neural Networks](https://mlsanity.com/paper/2009.02326/) (2020)
- [Untargeted, Targeted and Universal Adversarial Attacks and Defenses on Time Series](https://mlsanity.com/paper/2101.05639/) (2021)
- [Trojan Signatures in DNN Weights](https://mlsanity.com/paper/2109.02836/) (2021)
- [TOP: Backdoor Detection in Neural Networks via Transferability of Perturbation](https://mlsanity.com/paper/2103.10274/) (2021)
- [TAD: Trigger Approximation based Black-box Trojan Detection for AI](https://mlsanity.com/paper/2102.01815/) (2021)

## Built on by (most cited first, 1 of 1)

- [OCGEC: One-class Graph Embedding Classification for DNN Backdoor Detection](https://mlsanity.com/paper/2312.01585/) (2023)

## Abstract (arXiv)

With the surge of Machine Learning (ML), An emerging amount of intelligent applications have been developed. Deep Neural Networks (DNNs) have demonstrated unprecedented performance across various fields such as medical diagnosis and autonomous driving. While DNNs are widely employed in security-sensitive fields, they are identified to be vulnerable to Neural Trojan (NT) attacks that are controlled and activated by stealthy triggers. In this paper, we target to design a robust and adaptive Trojan detection scheme that inspects whether a pre-trained model has been Trojaned before its deployment. Prior works are oblivious of the intrinsic property of trigger distribution and try to reconstruct the trigger pattern using simple heuristics, i.e., stimulating the given model to incorrect outputs. As a result, their detection time and effectiveness are limited. We leverage the observation that the pixel trigger typically features spatial dependency and propose the first trigger approximation based black-box Trojan detection framework that enables a fast and scalable search of the trigger in the input space. Furthermore, our approach can also detect Trojans embedded in the feature space where certain filter transformations are used to activate the Trojan. We perform extensive experiments to investigate the performance of our approach across various datasets and ML models. Empirical results show that our approach achieves a ROC-AUC score of 0.93 on the public TrojAI dataset. Our code can be found at https://github.com/xinqiaozhang/adatrojan