# Attacks Meet Interpretability: Attribute-steered Detection of Adversarial Samples

arXiv: https://arxiv.org/abs/1810.11580 · HTML: https://mlsanity.com/paper/1810.11580/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Oct 2018.

## Built on by (most cited first, 30 of 38)

- [On Evaluating Adversarial Robustness of Large Vision-Language Models](https://mlsanity.com/paper/2305.16934/) (2023)
- [Advances in adversarial attacks and defenses in computer vision: A survey](https://mlsanity.com/paper/2108.00401/) (2021)
- [Robustness and Accuracy Could Be Reconcilable by (Proper) Definition](https://mlsanity.com/paper/2202.10103/) (2022)
- ["That Is a Suspicious Reaction!": Interpreting Logits Variation to Detect NLP Adversarial Attacks](https://mlsanity.com/paper/2204.04636/) (2022)
- [Securely Fine-tuning Pre-trained Encoders Against Adversarial Examples](https://mlsanity.com/paper/2403.10801/) (2024)
- [Predify: Augmenting deep neural networks with brain-inspired predictive coding dynamics](https://mlsanity.com/paper/2106.02749/) (2021)
- [Security and Privacy for Artificial Intelligence: Opportunities and Challenges](https://mlsanity.com/paper/2102.04661/) (2021)
- [Adversarial Imaging Pipelines](https://mlsanity.com/paper/2102.03728/) (2021)
- [Joint inference and input optimization in equilibrium networks](https://mlsanity.com/paper/2111.13236/) (2021)
- [AEON: A Method for Automatic Evaluation of NLP Test Cases](https://mlsanity.com/paper/2205.06439/) (2022)
- [Two Coupled Rejection Metrics Can Tell Adversarial Examples Apart](https://mlsanity.com/paper/2105.14785/) (2021)
- [Interpreting Attributions and Interactions of Adversarial Attacks](https://mlsanity.com/paper/2108.06895/) (2021)
- [As Firm As Their Foundations: Can open-sourced foundation models be used to create adversarial examples for downstream tasks?](https://mlsanity.com/paper/2403.12693/) (2024)
- [When and How to Fool Explainable Models (and Humans) with Adversarial Examples](https://mlsanity.com/paper/2107.01943/) (2021)
- [Efficient Adversarial Training With Data Pruning](https://mlsanity.com/paper/2207.00694/) (2022)
- [Beyond Boundaries: A Comprehensive Survey of Transferable Attacks on AI Systems](https://mlsanity.com/paper/2311.11796/) (2023)
- [Towards Variable-Length Textual Adversarial Attacks](https://mlsanity.com/paper/2104.08139/) (2021)
- [Investigating the significance of adversarial attacks and their relation to interpretability for radar-based human activity recognition systems](https://mlsanity.com/paper/2101.10562/) (2021)
- [Interpretations Cannot Be Trusted: Stealthy and Effective Adversarial Perturbations against Interpretable Deep Learning](https://mlsanity.com/paper/2211.15926/) (2022)
- [Transcending Adversarial Perturbations: Manifold-Aided Adversarial Examples with Legitimate Semantics](https://mlsanity.com/paper/2402.03095/) (2024)
- [A Survey on Physical Adversarial Attacks against Face Recognition Systems](https://mlsanity.com/paper/2410.16317/) (2024)
- [White Box Methods for Explanations of Convolutional Neural Networks in Image Classification Tasks](https://mlsanity.com/paper/2104.02548/) (2021)
- [PASA: Attack Agnostic Unsupervised Adversarial Detection using Prediction & Attribution Sensitivity Analysis](https://mlsanity.com/paper/2404.10789/) (2024)
- [Adversarial Examples Detection with Bayesian Neural Network](https://mlsanity.com/paper/2105.08620/) (2021)
- [Policy Distillation with Selective Input Gradient Regularization for Efficient Interpretability](https://mlsanity.com/paper/2205.08685/) (2022)
- [MEAD: A Multi-Armed Approach for Evaluation of Adversarial Examples Detectors](https://mlsanity.com/paper/2206.15415/) (2022)
- [A Random-patch based Defense Strategy Against Physical Attacks for Face Recognition Systems](https://mlsanity.com/paper/2304.07822/) (2023)
- [ViTGuard: Attention-aware Detection against Adversarial Examples for Vision Transformer](https://mlsanity.com/paper/2409.13828/) (2024)
- [i-Algebra: Towards Interactive Interpretability of Deep Neural Networks](https://mlsanity.com/paper/2101.09301/) (2021)
- [Downlink Power Allocation in Massive MIMO via Deep Learning: Adversarial Attacks and Training](https://mlsanity.com/paper/2206.06592/) (2022)

## Abstract (arXiv)

Adversarial sample attacks perturb benign inputs to induce DNN misbehaviors. Recent research has demonstrated the widespread presence and the devastating consequences of such attacks. Existing defense techniques either assume prior knowledge of specific attacks or may not work well on complex models due to their underlying assumptions. We argue that adversarial sample attacks are deeply entangled with interpretability of DNN models: while classification results on benign inputs can be reasoned based on the human perceptible features/attributes, results on adversarial samples can hardly be explained. Therefore, we propose a novel adversarial sample detection technique for face recognition models, based on interpretability. It features a novel bi-directional correspondence inference between attributes and internal neurons to identify neurons critical for individual attributes. The activation values of critical neurons are enhanced to amplify the reasoning part of the computation and the values of other neurons are weakened to suppress the uninterpretable part. The classification results after such transformation are compared with those of the original model to detect adversaries. Results show that our technique can achieve 94% detection accuracy for 7 different kinds of attacks with 9.91% false positives on benign inputs. In contrast, a state-of-the-art feature squeezing technique can only achieve 55% accuracy with 23.3% false positives.