mlsanityData to 7 Oct 2026

MISA: Online Defense of Trojaned Models using Misattributions

arXiv 2103.15918 ↗First posted 29 Mar 2021cs.CR cs.CV stat.MLcited by 4

Not on the map. Why?

Abstract (arXiv)

Recent studies have shown that neural networks are vulnerable to Trojan attacks, where a network is trained to respond to specially crafted trigger patterns in the inputs in specific and potentially malicious ways. This paper proposes MISA, a new online approach to detect Trojan triggers for neural networks at inference time. Our approach is based on a novel notion called misattributions, which captures the anomalous manifestation of a Trojan activation in the feature space. Given an input image and the corresponding output prediction, our algorithm first computes the model's attribution on different features. It then statistically analyzes these attributions to ascertain the presence of a Trojan trigger. Across a set of benchmarks, we show that our method can effectively detect Trojan triggers for a wide variety of trigger patterns, including several recent ones for which there are no known defenses. Our method achieves 96% AUC for detecting images that include a Trojan trigger without any assumptions on the trigger pattern.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 69 of its 87 references that are in the map. First posted on arXiv 29 Mar 2021.

Builds on

69 of 87 references.

References · 69

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  4. 2013Intriguing properties of neural networkscited by 5,114
  5. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  6. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  7. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  8. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  9. 2019EfficientDet: Scalable and Efficient Object Detectioncited by 1,018
  10. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  11. 2016End to End Learning for Self-Driving Carscited by 1,000
  12. 2014Striving for Simplicity: The All Convolutional Netcited by 983
38 more
  1. 2016Universal adversarial perturbationscited by 774
  2. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 703
  3. 2018MINE: Mutual Information Neural Estimationcited by 680
  4. 2018Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNscited by 585
  5. 2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 550
  6. 2018Spectral Signatures in Backdoor Attackscited by 457
  7. 2019STRIP: A Defence Against Trojan Attacks on Deep Neural Networkscited by 437
  8. 2018Detecting Backdoor Attacks on Deep Neural Networks by Activation Clusteringcited by 429
  9. 2020Backdoor Learning: A Surveycited by 412
  10. 2019Hidden Trigger Backdoor Attackscited by 359
  11. 2020Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networkscited by 341
  12. 2021Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networkscited by 313
  13. 2019Label-Consistent Backdoor Attackscited by 285
  14. 2019Fooling automated surveillance cameras: adversarial patches to attack person detectioncited by 268
  15. 2019Adversarial Attacks and Defenses in Images, Graphs and Text: A Reviewcited by 234
  16. 2019Detecting AI Trojans Using Meta Neural Analysiscited by 192
  17. 2020Blind Backdoors in Deep Learning Modelscited by 174
  18. 2017Neural Trojanscited by 151
  19. 2018SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systemscited by 148
  20. 2020Dynamic Backdoor Attacks Against Machine Learning Modelscited by 147
  21. 2019Universal Litmus Patterns: Revealing Backdoor Attacks in CNNscited by 143
  22. 2020Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustnesscited by 130
  23. 2020Backdoor Attacks Against Deep Learning Systems in the Physical Worldcited by 129
  24. 2020Backdoor Attacks to Graph Neural Networkscited by 126
  25. 2019TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systemscited by 120
  26. 2019XRAI: Better Attributions Through Regionscited by 116
  27. 2019TBT: Targeted Neural Network Attack with Bit Trojancited by 112
  28. 2020Just How Toxic is Data Poisoning? A Unified Benchmark for Backdoor and Data Poisoning Attackscited by 109
  29. 2018Adversarial Attacks and Defences Competitioncited by 96
  30. 2021Backdoor Scanning for Deep Neural Networks through K-Arm Optimizationcited by 96
  31. 2019Robust Anomaly Detection and Backdoor Attack Detection Via Differential Privacycited by 93
  32. 2020Practical Detection of Trojan Neural Networks: Data-Limited and Data-Free Casescited by 93
  33. 2020An Embarrassingly Simple Approach for Trojan Attack in Deep Neural Networkscited by 92
  34. 2018Generalizable Data-free Objective for Crafting Universal Adversarial Perturbationscited by 88
  35. 2020Strong Data Augmentation Sanitizes Poisoning and Backdoor Attacks Without an Accuracy Tradeoffcited by 86
  36. 2019Defending Neural Backdoors via Generative Distribution Modelingcited by 82
  37. 2020Rethinking the Trigger of Backdoor Attackcited by 76
  38. 2020DeepSweep: An Evaluation Framework for Mitigating DNN Backdoor Attacks using Data Augmentationcited by 75

50 most cited of 69 All 69 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 3; 2024: 1.

4 papers.

  1. 2023BEAGLE: Forensics of Deep Learning Backdoor Attack for Better Defensecited by 15
  2. 2023TIJO: Trigger Inversion with Joint Optimization for Defending Multimodal Backdoored Modelscited by 11
  3. 2024Protecting against simultaneous data poisoning attackscited by 1
  4. 2023Threats, Vulnerabilities, and Controls of Machine Learning Based Systems: A Survey and Taxonomycited by 0