mlsanityData to 7 Oct 2026

Eliminating Backdoor Triggers for Deep Neural Networks Using Attention Relation Graph Distillation

arXiv 2204.09975 ↗First posted 21 Apr 2022cs.LGcited by 18

Not on the map. Why?

Abstract (arXiv)

Due to the prosperity of Artificial Intelligence (AI) techniques, more and more backdoors are designed by adversaries to attack Deep Neural Networks (DNNs).Although the state-of-the-art method Neural Attention Distillation (NAD) can effectively erase backdoor triggers from DNNs, it still suffers from non-negligible Attack Success Rate (ASR) together with lowered classification ACCuracy (ACC), since NAD focuses on backdoor defense using attention features (i.e., attention maps) of the same order. In this paper, we introduce a novel backdoor defense framework named Attention Relation Graph Distillation (ARGD), which fully explores the correlation among attention features with different orders using our proposed Attention Relation Graphs (ARGs). Based on the alignment of ARGs between both teacher and student models during knowledge distillation, ARGD can eradicate more backdoor triggers than NAD. Comprehensive experimental results show that, against six latest backdoor attacks, ARGD outperforms NAD by up to 94.85% reduction in ASR, while ACC can be improved by up to 3.23%.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 20 of its 30 references that are in the map. First posted on arXiv 21 Apr 2022.

Builds on

20 of 30 references.

References · 20

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  3. 2016Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfercited by 1,059
  4. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  5. 2018How To Backdoor Federated Learningcited by 745
  6. 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 675
  7. 2018Spectral Signatures in Backdoor Attackscited by 457
  8. 2018Detecting Backdoor Attacks on Deep Neural Networks by Activation Clusteringcited by 429
  9. 2020Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networkscited by 341
  10. 2018Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooringcited by 334
  11. 2021Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networkscited by 313
  12. 2018SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systemscited by 148
8 more
  1. 2018Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbationcited by 131
  2. 2020Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustnesscited by 130
  3. 2020Backdoor Attacks Against Deep Learning Systems in the Physical Worldcited by 129
  4. 2019Adversarially Robust Distillationcited by 124
  5. 2020Rethinking the Trigger of Backdoor Attackcited by 76
  6. 2019Pay attention to the activations: a modular attention mechanism for fine-grained image recognitioncited by 12
  7. 2020FDA3 : Federated Defense Against Adversarial Attacks for Cloud-Based IIoT Applicationscited by 10
  8. 2020Interpreting and Disentangling Feature Components of Various Complexity from DNNscited by 8

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 8; 2024: 9.

18 papers.

Sort:

By area:

  1. 2023Label Poisoning is All You Needcited by 27
  2. 2022Backdoor Cleansing with Unlabeled Datacited by 17
  3. 2024BackdoorBench: A Comprehensive Benchmark and Analysis of Backdoor Learningcited by 17
  4. 2024Understanding the Gains from Repeated Self-Distillationcited by 14
  5. 2023WaveAttack: Asymmetric Frequency Obfuscation-based Backdoor Attacks Against Deep Neural Networkscited by 8
  6. 2024Mutual Information Guided Backdoor Mitigation for Pre-trained Encoderscited by 8
  7. 2023Leveraging Diffusion-Based Image Variations for Robust Training on Poisoned Datacited by 6
  8. 2023Privacy Inference-Empowered Stealthy Backdoor Attack on Federated Learning under Non-IID Scenarioscited by 3
  9. 2023Temporal-Distributed Backdoor Attack Against Video Based Action Recognitioncited by 3
  10. 2023INK: Inheritable Natural Backdoor Attack Against Model Distillationcited by 2
  11. 2024Robust Knowledge Distillation Based on Feature Variance Against Backdoored Teacher Modelcited by 2
  12. 2023Backdoor Mitigation by Correcting the Distribution of Neural Activationscited by 1
6 more
  1. 2024Invisible Backdoor Attack Through Singular Value Decompositioncited by 1
  2. 2024Prompting the Unseen: Detecting Hidden Backdoors in Black-Box Modelscited by 1
  3. 2024Trojan Cleansing with Neural Collapsecited by 1
  4. 2024BadScan: An Architectural Backdoor Attack on Visual State Space Modelscited by 1
  5. 2023SATBA: An Invisible Backdoor Attack Based On Spatial Attentioncited by 0
  6. 2024BackdoorBench: A Comprehensive Benchmark and Analysis of Backdoor Learningcited by 0