# Cassandra: Detecting Trojaned Networks from Adversarial Perturbations

arXiv: https://arxiv.org/abs/2007.14433 · HTML: https://mlsanity.com/paper/2007.14433/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jul 2020.

## Built on by (most cited first, 15 of 15)

- [Backdoor Scanning for Deep Neural Networks through K-Arm Optimization](https://mlsanity.com/paper/2102.05123/) (2021)
- [Fine-mixing: Mitigating Backdoors in Fine-tuned Language Models](https://mlsanity.com/paper/2210.09545/) (2022)
- [BAGM: A Backdoor Attack for Manipulating Text-to-Image Generative Models](https://mlsanity.com/paper/2307.16489/) (2023)
- [How to Inject Backdoors with Better Consistency: Logit Anchoring on Clean Data](https://mlsanity.com/paper/2109.01300/) (2021)
- [Poser: Unmasking Alignment Faking LLMs by Manipulating Their Internals](https://mlsanity.com/paper/2405.05466/) (2024)
- [TIJO: Trigger Inversion with Joint Optimization for Defending Multimodal Backdoored Models](https://mlsanity.com/paper/2308.03906/) (2023)
- [SSL-Cleanse: Trojan Detection and Mitigation in Self-Supervised Learning](https://mlsanity.com/paper/2303.09079/) (2023)
- [Dim-Krum: Backdoor-Resistant Federated Learning for NLP with Dimension-wise Krum-Based Aggregation](https://mlsanity.com/paper/2210.06894/) (2022)
- [Detecting Backdoor in Deep Neural Networks via Intentional Adversarial Perturbations](https://mlsanity.com/paper/2105.14259/) (2021)
- [Realizable Universal Adversarial Perturbations for Malware](https://mlsanity.com/paper/2102.06747/) (2021)
- [EX-RAY: Distinguishing Injected Backdoor from Natural Features in Neural Networks by Examining Differential Feature Symmetry](https://mlsanity.com/paper/2103.08820/) (2021)
- [Adversarially Guided Stateful Defense Against Backdoor Attacks in Federated Deep Learning](https://mlsanity.com/paper/2410.11205/) (2024)
- [MISA: Online Defense of Trojaned Models using Misattributions](https://mlsanity.com/paper/2103.15918/) (2021)
- [DECK: Model Hardening for Defending Pervasive Backdoors](https://mlsanity.com/paper/2206.09272/) (2022)
- [Scanning Trojaned Models Using Out-of-Distribution Samples](https://mlsanity.com/paper/2501.17151/) (2025)

## Abstract (arXiv)

Deep neural networks are being widely deployed for many critical tasks due to their high classification accuracy. In many cases, pre-trained models are sourced from vendors who may have disrupted the training pipeline to insert Trojan behaviors into the models. These malicious behaviors can be triggered at the adversary's will and hence, cause a serious threat to the widespread deployment of deep models. We propose a method to verify if a pre-trained model is Trojaned or benign. Our method captures fingerprints of neural networks in the form of adversarial perturbations learned from the network gradients. Inserting backdoors into a network alters its decision boundaries which are effectively encoded in their adversarial perturbations. We train a two stream network for Trojan detection from its global (L_∞ and L2 bounded) perturbations and the localized region of high energy within each perturbation. The former encodes decision boundaries of the network and latter encodes the unknown trigger shape. We also propose an anomaly detection method to identify the target class in a Trojaned network. Our methods are invariant to the trigger type, trigger size, training data and network architecture. We evaluate our methods on MNIST, NIST-Round0 and NIST-Round1 datasets, with up to 1,000 pre-trained models making this the largest study to date on Trojaned network detection, and achieve over 92% detection accuracy to set the new state-of-the-art.