# CatchBackdoor: Backdoor Detection via Critical Trojan Neural Path Fuzzing

arXiv: https://arxiv.org/abs/2112.13064 · HTML: https://mlsanity.com/paper/2112.13064/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 44 references that are in the map.
- First posted on arXiv 24 Dec 2021.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017)
- [Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networks](https://mlsanity.com/paper/1804.00792/) (2018)
- [Spectral Signatures in Backdoor Attacks](https://mlsanity.com/paper/1811.00636/) (2018)
- [STRIP: A Defence Against Trojan Attacks on Deep Neural Networks](https://mlsanity.com/paper/1902.06531/) (2019)
- [Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering](https://mlsanity.com/paper/1811.03728/) (2018)
- [Hidden Trigger Backdoor Attacks](https://mlsanity.com/paper/1910.00033/) (2019)
- [Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks](https://mlsanity.com/paper/2007.02343/) (2020)
- [A new Backdoor Attack in CNNs by training set corruption without label poisoning](https://mlsanity.com/paper/1902.11237/) (2019)
- [Adversarial Neuron Pruning Purifies Backdoored Deep Models](https://mlsanity.com/paper/2110.14430/) (2021)
- [Detecting AI Trojans Using Meta Neural Analysis](https://mlsanity.com/paper/1910.03137/) (2019)
- [Dynamic Backdoor Attacks Against Machine Learning Models](https://mlsanity.com/paper/2003.03675/) (2020)
- [Universal Litmus Patterns: Revealing Backdoor Attacks in CNNs](https://mlsanity.com/paper/1906.10842/) (2019)
- [Demon in the Variant: Statistical Analysis of DNNs for Robust Backdoor Contamination Detection](https://mlsanity.com/paper/1908.00686/) (2019)
- [TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systems](https://mlsanity.com/paper/1908.01763/) (2019)
- [Sleeper Agent: Scalable Hidden Trigger Backdoors for Neural Networks Trained from Scratch](https://mlsanity.com/paper/2106.08970/) (2021)
- [Backdoor Scanning for Deep Neural Networks through K-Arm Optimization](https://mlsanity.com/paper/2102.05123/) (2021)
- [Practical Detection of Trojan Neural Networks: Data-Limited and Data-Free Cases](https://mlsanity.com/paper/2007.15802/) (2020)
- [Deep Feature Space Trojan Attack of Neural Networks by Controlled Detoxification](https://mlsanity.com/paper/2012.11212/) (2020)
- [Bypassing Backdoor Detection Algorithms in Deep Learning](https://mlsanity.com/paper/1905.13409/) (2019)
- [Bullseye Polytope: A Scalable Clean-Label Poisoning Attack with Improved Transferability](https://mlsanity.com/paper/2005.00191/) (2020)
- [Model Agnostic Defence against Backdoor Attacks in Machine Learning](https://mlsanity.com/paper/1908.02203/) (2019)
- [DeepPayload: Black-box Backdoor Attack on Deep Learning Models through Neural Payload Injection](https://mlsanity.com/paper/2101.06896/) (2021)
- [Trigger Hunting with a Topological Prior for Trojan Detection](https://mlsanity.com/paper/2110.08335/) (2021)

## Built on by (most cited first, 1 of 1)

- [PSBD: Prediction Shift Uncertainty Unlocks Backdoor Detection](https://mlsanity.com/paper/2406.05826/) (2024)

## Abstract (arXiv)

The success of deep neural networks (DNNs) in real-world applications has benefited from abundant pre-trained models. However, the backdoored pre-trained models can pose a significant trojan threat to the deployment of downstream DNNs. Numerous backdoor detection methods have been proposed but are limited to two aspects: (1) high sensitivity on trigger size, especially on stealthy attacks (i.e., blending attacks and defense adaptive attacks); (2) rely heavily on benign examples for reverse engineering. To address these challenges, we empirically observed that trojaned behaviors triggered by various trojan attacks can be attributed to the trojan path, composed of top-k critical neurons with more significant contributions to model prediction changes. Motivated by it, we propose CatchBackdoor, a detection method against trojan attacks. Based on the close connection between trojaned behaviors and trojan path to trigger errors, CatchBackdoor starts from the benign path and gradually approximates the trojan path through differential fuzzing. We then reverse triggers from the trojan path, to trigger errors caused by diverse trojaned attacks. Extensive experiments on MINST, CIFAR-10, and a-ImageNet datasets and 7 models (LeNet, ResNet, and VGG) demonstrate the superiority of CatchBackdoor over the state-of-the-art methods, in terms of (1) effective - it shows better detection performance, especially on stealthy attacks (~ × 2 on average); (2) extensible - it is robust to trigger size and can conduct detection without benign examples.