mlsanityData to 14 Oct 2026

Adversarial Attacks on the Interpretation of Neuron Activation Maximization

arXiv 2306.07397 ↗First posted 12 Jun 2023cs.LG cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

The internal functional behavior of trained Deep Neural Networks is notoriously difficult to interpret. Activation-maximization approaches are one set of techniques used to interpret and analyze trained deep-learning models. These consist in finding inputs that maximally activate a given neuron or feature map. These inputs can be selected from a data set or obtained by optimization. However, interpretability methods may be subject to being deceived. In this work, we consider the concept of an adversary manipulating a model for the purpose of deceiving the interpretation. We propose an optimization framework for performing this manipulation and demonstrate a number of ways that popular activation-maximization interpretation techniques associated with CNNs can be manipulated to change the interpretations, shedding light on the reliability of these methods.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 43 of its 61 references that are in the map. First posted on arXiv 12 Jun 2023.

Builds on

43 of 61 references.

References · 43

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  5. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  6. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  7. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  8. 2020Scaling Laws for Neural Language Modelscited by 6,586
  9. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  10. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  11. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  12. 2013Visualizing and Understanding Convolutional Networkscited by 2,290
31 more
  1. 2016Equality of Opportunity in Supervised Learningcited by 2,007
  2. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,917
  3. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  4. 2020Concept Bottleneck Modelscited by 1,032
  5. 2023Progress measures for grokking via mechanistic interpretabilitycited by 839
  6. 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
  7. 2019Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methodscited by 443
  8. 2014Understanding Deep Image Representations by Inverting Themcited by 403
  9. 2015Understanding Neural Networks Through Deep Visualizationcited by 334
  10. 2019Counterfactual Visual Explanationscited by 313
  11. 2020Concept Whitening for Interpretable Image Recognitioncited by 225
  12. 2019Explanations can be manipulated and geometry is to blamecited by 216
  13. 2018Fair Clustering Through Fairletscited by 176
  14. 2017Interpretable Convolutional Neural Networkscited by 162
  15. 2019Towards Fairer Datasets: Filtering and Balancing the Distribution of the People Subtree in the ImageNet Hierarchycited by 156
  16. 2022CLIP-Dissect: Automatic Description of Neuron Representations in Deep Vision Networkscited by 147
  17. 2022On Feature Learning in the Presence of Spurious Correlationscited by 145
  18. 2022Natural Language Descriptions of Deep Visual Featurescited by 137
  19. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 129
  20. 2013Stabilitycited by 120
  21. 2019Fooling Neural Network Interpretations via Adversarial Model Manipulationcited by 112
  22. 2022Reliability of CKA as a Similarity Measure in Deep Learningcited by 87
  23. 2020Fairwashing Explanations with Off-Manifold Detergentcited by 72
  24. 2021"Will You Find These Shortcuts?" A Protocol for Evaluating the Faithfulness of Input Salience Methods for Text Classificationcited by 66
  25. 2021Entropy-based Logic Explanations of Neural Networkscited by 51
  26. 2021Shapley Explanation Networkscited by 41
  27. 2021How Well do Feature Visualizations Support Causal Understanding of CNN Activations?cited by 28
  28. 2020A Framework to Learn with Interpretationcited by 23
  29. 2021Characterizing the risk of fairwashingcited by 21
  30. 2022Conformalized Fairness via Quantile Regressioncited by 19
  31. 2021Interpretable Compositional Convolutional Neural Networkscited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 5; 2025: 0; 2026: 1.

7 papers.

  1. 2024Linear Explanations for Individual Neuronscited by 19
  2. 2023Neuron Sensitivity Guided Test Case Selection for Deep Learning Testingcited by 3
  3. 2024Manipulating Feature Visualizations with Gradient Slingshotscited by 3
  4. 2024Attack Anything: Blind DNNs via Universal Background Adversarial Attackcited by 3
  5. 2024Privacy and Security Implications of Cloud-Based AI Services : A Surveycited by 0
  6. 2024From Feature Visualization to Visual Circuits: Effect of Adversarial Model Manipulationcited by 0
  7. 2026LINE: LLM-based Iterative Neuron Explanations for Vision Modelscited by 0