mlsanityData to 7 Oct 2026

Explainable Adversarial Attacks in Deep Neural Networks Using Activation Profiles

arXiv 2103.10229 ↗First posted 18 Mar 2021cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

As neural networks become the tool of choice to solve an increasing variety of problems in our society, adversarial attacks become critical. The possibility of generating data instances deliberately designed to fool a network's analysis can have disastrous consequences. Recent work has shown that commonly used methods for model training often result in fragile abstract representations that are particularly vulnerable to such attacks. This paper presents a visual framework to investigate neural network models subjected to adversarial examples, revealing how models' perception of the adversarial data differs from regular data instances and their relationships with class perception. Through different use cases, we show how observing these elements can quickly pinpoint exploited areas in a model, allowing further study of vulnerable features in input data and serving as a guide to improving model training and architecture.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 52 of its 91 references that are in the map. First posted on arXiv 18 Mar 2021.

Builds on

52 of 91 references.

References · 52

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2013Intriguing properties of neural networkscited by 5,114
  5. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  7. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  8. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  9. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  10. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  11. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,315
  12. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
38 more
  1. 2020Fast is better than free: Revisiting adversarial trainingcited by 723
  2. 2019Adversarial Training for Free!cited by 638
  3. 2017One pixel attack for fooling deep neural networkscited by 579
  4. 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 542
  5. 2018Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Surveycited by 459
  6. 2019STRIP: A Defence Against Trojan Attacks on Deep Neural Networkscited by 437
  7. 2015Evaluating the visualization of what a Deep Neural Network has learnedcited by 416
  8. 2017Wild Patterns: Ten Years After the Rise of Adversarial Machine Learningcited by 411
  9. 2019A Survey of Deep Learning Techniques for Autonomous Drivingcited by 401
  10. 2019Hidden Trigger Backdoor Attackscited by 359
  11. 2017Adversarial Examples: Attacks and Defenses for Deep Learningcited by 350
  12. 2015Understanding Neural Networks Through Deep Visualizationcited by 332
  13. 2018Adversarial Attacks and Defences: A Surveycited by 259
  14. 2018Generating Adversarial Examples with Adversarial Networkscited by 256
  15. 2015Visualizing and Understanding Neural Models in NLPcited by 199
  16. 2017Towards Poisoning of Deep Learning Algorithms with Back-gradient Optimizationcited by 188
  17. 2015Visualizing and Understanding Recurrent Networkscited by 158
  18. 2018Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbationcited by 131
  19. 2018Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontierscited by 117
  20. 2016Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networkscited by 106
  21. 2017Learning how to explain neural networks: PatternNet and PatternAttributioncited by 84
  22. 2019Summit: Scaling Deep Learning Interpretability by Visualizing Activation and Attribution Summarizationscited by 72
  23. 2018RetainVis: Visual Analytics with Interpretable and Interactive Recurrent Neural Networks on Electronic Medical Recordscited by 61
  24. 2020CNN Explainer: Learning Convolutional Neural Networks with Interactive Visualizationcited by 57
  25. 2017ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Modelscited by 56
  26. 2016Towards Better Analysis of Deep Convolutional Neural Networkscited by 51
  27. 2017Towards better understanding of gradient-based attribution methods for Deep Neural Networkscited by 51
  28. 2018Manifold: A Model-Agnostic Framework for Interpretation and Diagnosis of Machine Learning Modelscited by 50
  29. 2017Understanding Hidden Memories of Recurrent Neural Networkscited by 43
  30. 2018GAN Lab: Understanding Complex Deep Generative Models using Interactive Visual Experimentationcited by 40
  31. 2018Visual Analytics for Explainable Deep Learningcited by 39
  32. 2017Towards Better Analysis of Machine Learning Models: A Visual Analytics Perspectivecited by 36
  33. 2020Analyzing the Noise Robustness of Deep Neural Networkscited by 31
  34. 2019Explaining Vulnerabilities to Adversarial Machine Learning through Visual Analyticscited by 23
  35. 2017A Workflow for Visual Diagnostics of Binary Classifiers using Instance-Level Explanationscited by 20
  36. 2017Adversarial Robustness: Softmax versus Openmaxcited by 7
  37. 2018DeepTracker: Visualizing the Training Process of Convolutional Neural Networkscited by 7
  38. 2017An Introduction to Deep Visual Explanationcited by 4

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 2; 2023: 0; 2024: 2; 2025: 1.

6 papers.

  1. 2022Generalizing Adversarial Explanations with Grad-CAMcited by 9
  2. 2022VisCUIT: Visual Auditor for Bias in CNN Image Classifiercited by 8
  3. 2024EvolvED: Evolutionary Embeddings to Understand the Generation Process of Diffusion Modelscited by 3
  4. 2025The Structural Safety Generalization Problemcited by 2
  5. 2021TESDA: Transform Enabled Statistical Detection of Attacks in Deep Neural Networkscited by 0
  6. 2024Searching for internal symbols underlying deep learningcited by 0