mlsanityData to 14 Oct 2026

A Survey on Understanding, Visualizations, and Explanation of Deep Neural Networks

arXiv 2102.01792 ↗First posted 2 Feb 2021cs.LG cs.AIcited by 14

Not on the map. Why?

Abstract (arXiv)

Recent advancements in machine learning and signal processing domains have resulted in an extensive surge of interest in Deep Neural Networks (DNNs) due to their unprecedented performance and high accuracy for different and challenging problems of significant engineering importance. However, when such deep learning architectures are utilized for making critical decisions such as the ones that involve human lives (e.g., in control systems and medical applications), it is of paramount importance to understand, trust, and in one word "explain" the argument behind deep models' decisions. In many applications, artificial neural networks (including DNNs) are considered as black-box systems, which do not provide sufficient clue on their internal processing actions. Although some recent efforts have been initiated to explain the behaviors and decisions of deep networks, explainable artificial intelligence (XAI) domain, which aims at reasoning about the behavior and decisions of DNNs, is still in its infancy. The aim of this paper is to provide a comprehensive overview on Understanding, Visualization, and Explanation of the internal and overall behavior of DNNs.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 53 of its 92 references that are in the map. First posted on arXiv 2 Feb 2021.

Builds on

53 of 92 references.

References · 53

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2016Densely Connected Convolutional Networkscited by 6,720
  4. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  5. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  6. 2014Going Deeper with Convolutionscited by 4,241
  7. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  8. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,694
  9. 2015Learning Deep Features for Discriminative Localizationcited by 2,436
  10. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  11. 2013Visualizing and Understanding Convolutional Networkscited by 2,290
  12. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
38 more
  1. 2017Learning Important Features Through Propagating Activation Differencescited by 1,458
  2. 2014How transferable are features in deep neural networks?cited by 1,225
  3. 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,198
  4. 2017SmoothGrad: removing noise by adding noisecited by 1,109
  5. 2016End to End Learning for Self-Driving Carscited by 1,009
  6. 2018A Survey Of Methods For Explaining Black Box Modelscited by 973
  7. 2014Deep Visual-Semantic Alignments for Generating Image Descriptionscited by 968
  8. 2016The Mythos of Model Interpretabilitycited by 721
  9. 2017Dynamic Routing Between Capsulescited by 689
  10. 2014Understanding Deep Image Representations by Inverting Themcited by 403
  11. 2017Methods for Interpreting and Understanding Deep Neural Networkscited by 345
  12. 2016A Learned Representation For Artistic Stylecited by 341
  13. 2015Understanding Neural Networks Through Deep Visualizationcited by 334
  14. 2015LSTM: A Search Space Odysseycited by 316
  15. 2018Explanations based on the Missing: Towards Contrastive Explanations with Pertinent Negativescited by 262
  16. 2016Understanding Neural Networks through Representation Erasurecited by 224
  17. 2017Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Modelscited by 206
  18. 2015Visualizing and Understanding Neural Models in NLPcited by 200
  19. 2015Semi-supervised Sequence Learningcited by 198
  20. 2018Representer Point Selection for Explaining Deep Neural Networkscited by 187
  21. 2016Synthesizing the preferred inputs for neurons in neural networks via deep generator networkscited by 181
  22. 2016Texture Networks: Feed-forward Synthesis of Textures and Stylized Imagescited by 174
  23. 2017Visualizing Deep Neural Network Decisions: Prediction Difference Analysiscited by 166
  24. 2017Interpretable Convolutional Neural Networkscited by 162
  25. 2015Visualizing and Understanding Recurrent Networkscited by 160
  26. 2018Visual Interpretability for Deep Learning: a Surveycited by 155
  27. 2015Brain Tumor Segmentation with Deep Neural Networkscited by 137
  28. 2015Character-Aware Neural Language Modelscited by 132
  29. 2018Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontierscited by 117
  30. 2016Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networkscited by 106
  31. 2015Visualizing Deep Convolutional Neural Networks Using Natural Pre-Imagescited by 93
  32. 2018Interpreting CNNs via Decision Treescited by 89
  33. 2018Interpretable Convolutional Neural Networks via Feedforward Designcited by 49
  34. 2017Understanding Hidden Memories of Recurrent Neural Networkscited by 44
  35. 2016Representation of linguistic form and function in recurrent neural networkscited by 39
  36. 2016Gradients of Counterfactualscited by 31
  37. 2017Denoising Adversarial Autoencoderscited by 20
  38. 2018Explaining Deep Learning Models - A Bayesian Non-parametric Approachcited by 16

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 3; 2022: 2; 2023: 2; 2024: 5; 2025: 1; 2026: 1.

14 papers.

Sort:

By area:

  1. 2024A Cross-Language Investigation into Jailbreak Attacks in Large Language Modelscited by 41
  2. 2024Compute Optimal Inference and Provable Amortisation Gap in Sparse Autoencoderscited by 18
  3. 2024Explainable AI (XAI) in Image Segmentation in Medicine, Industry, and Beyond: A Surveycited by 11
  4. 2021Spectral Transform Forms Scalable Transformercited by 4
  5. 2022A Fast Attention Network for Joint Intent Detection and Slot Filling on Edge Devicescited by 3
  6. 2023A Brief Review of Explainable Artificial Intelligence in Healthcarecited by 2
  7. 2024Leaky ReLUs That Differ in Forward and Backward Pass Facilitate Activation Maximization in Deep Neural Networkscited by 2
  8. 2021Silhouettes and quasi residual plots for neural nets and tree-based classifierscited by 1
  9. 2022The Influence of Explainable Artificial Intelligence: Nudging Behaviour or Boosting Capability?cited by 1
  10. 2024Understanding Neural Network Systems for Image Analysis using Vector Spaces and Inverse Mapscited by 1
  11. 2025Explainable embeddings with Distance Explainercited by 1
  12. 2021Feature Alignment as a Generative Processcited by 0
2 more
  1. 2023Explaining Deep Face Algorithms through Visualization: A Surveycited by 0
  2. 2026Evaluating Contextual Bias in CNN Image Classification: Evidence from Agricultural Benchmark Datasetscited by 0