mlsanityData to 7 Oct 2026

Interpreting Attributions and Interactions of Adversarial Attacks

arXiv 2108.06895 ↗First posted 16 Aug 2021cs.LG cs.AI cs.CVcited by 13

Not on the map. Why?

Abstract (arXiv)

This paper aims to explain adversarial attacks in terms of how adversarial perturbations contribute to the attacking task. We estimate attributions of different image regions to the decrease of the attacking cost based on the Shapley value. We define and quantify interactions among adversarial perturbation pixels, and decompose the entire perturbation map into relatively independent perturbation components. The decomposition of the perturbation map shows that adversarially-trained DNNs have more perturbation components in the foreground than normally-trained DNNs. Moreover, compared to the normally-trained DNN, the adversarially-trained DNN have more components which mainly decrease the score of the true category. Above analyses provide new insights into the understanding of adversarial attacks.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 44 of its 61 references that are in the map. First posted on arXiv 16 Aug 2021.

Builds on

44 of 61 references.

References · 44

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  6. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  7. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  8. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  9. 2013Intriguing properties of neural networkscited by 5,114
  10. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  11. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  12. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
32 more
  1. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  2. 2016Adversarial examples in the physical worldcited by 1,762
  3. 2018Sanity Checks for Saliency Mapscited by 1,149
  4. 2014Striving for Simplicity: The All Convolutional Netcited by 983
  5. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
  6. 2016Practical Black-Box Attacks against Machine Learningcited by 893
  7. 2018Robustness May Be at Odds with Accuracycited by 873
  8. 2017Ensemble Adversarial Training: Attacks and Defensescited by 833
  9. 2015The Limitations of Deep Learning in Adversarial Settingscited by 784
  10. 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 675
  11. 2017ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Modelscited by 656
  12. 2017One pixel attack for fooling deep neural networkscited by 579
  13. 2016Delving into Transferable Adversarial Examples and Black-box Attackscited by 530
  14. 2017Detecting Adversarial Samples from Artifactscited by 244
  15. 2017On Detecting Adversarial Perturbationscited by 232
  16. 2018Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approachcited by 181
  17. 2016Improving the Robustness of Deep Neural Networks via Stability Trainingcited by 156
  18. 2020Explaining Explanations: Axiomatic Feature Interactions for Deep Networkscited by 103
  19. 2018L-Shapley and C-Shapley: Efficient Model Interpretation for Structured Datacited by 100
  20. 2017Detecting Statistical Interactions from Neural Network Weightscited by 80
  21. 2018Adversarial examples from computational constraintscited by 79
  22. 2017SafetyNet: Detecting and Rejecting Adversarial Examples Robustlycited by 76
  23. 2020A Unified Approach to Interpreting and Boosting Adversarial Transferabilitycited by 71
  24. 2018Beyond Word Importance: Contextual Decomposition to Extract Interactions from LSTMscited by 67
  25. 2018Hierarchical interpretations for neural network predictionscited by 66
  26. 2018Structured Adversarial Attack: Towards General Implementation and Better Interpretabilitycited by 63
  27. 2018Attacks Meet Interpretability: Attribute-steered Detection of Adversarial Samplescited by 38
  28. 2020Interpreting and Boosting Dropout from a Game-Theoretic Viewcited by 34
  29. 2019Towards Hierarchical Importance Attribution: Explaining Compositional Semantics for Neural Sequence Modelscited by 26
  30. 2017Consistent feature attribution for tree ensemblescited by 17
  31. 2020Technical Note: Game-Theoretic Interactions of Different Orderscited by 11
  32. 2020Interpreting Multivariate Shapley Interactions in DNNscited by 4

Built on by

Cited by year

Citing papers per year: 2021: 4; 2022: 0; 2023: 4; 2024: 2; 2025: 3.

13 papers.

Sort:

By area:

  1. 2021Discovering and Explaining the Representation Bottleneck of DNNscited by 42
  2. 2021Defining and Quantifying the Emergence of Sparse Concepts in DNNscited by 36
  3. 2023Does a Neural Network Really Encode Symbolic Concepts?cited by 27
  4. 2021A Unified Game-Theoretic Interpretation of Adversarial Robustnesscited by 17
  5. 2023Can the Inference Logic of Large Language Models be Disentangled into Symbolic Concepts?cited by 11
  6. 2023Technical Note: Defining and Quantifying AND-OR Interactions for Faithful and Concise Explanation of DNNscited by 10
  7. 2025INTER: Mitigating Hallucination in Large Vision-Language Models by Interaction Guidance Samplingcited by 10
  8. 2021Visualizing the Emergence of Intermediate Visual Patterns in DNNscited by 7
  9. 2024Diagnosing the Compositional Knowledge of Vision Language Models from a Game-Theoretic Viewcited by 7
  10. 2024Two-Phase Dynamics of Interactions Explains the Starting Point of a DNN Learning Over-Fitted Featurescited by 5
  11. 2023Good-looking but Lacking Faithfulness: Understanding Local Explanation Methods through Trend-based Testingcited by 2
  12. 2025The Interaction Bottleneck of Deep Neural Networks: Discovery, Proof, and Modulationcited by 1
1 more
  1. 2025Adversarial Wear and Tear: Exploiting Natural Damage for Generating Physical-World Adversarial Examplescited by 0