mlsanityData to 7 Oct 2026

Visual Interpretability for Deep Learning: a Survey

arXiv 1802.00614 ↗First posted 2 Feb 2018cs.CVcited by 155

Not on the map. Why?

Abstract (arXiv)

This paper reviews recent studies in understanding neural-network representations and learning neural networks with interpretable/disentangled middle-layer representations. Although deep neural networks have exhibited superior performance in various tasks, the interpretability is always the Achilles' heel of deep neural networks. At present, deep neural networks obtain high discrimination power at the cost of low interpretability of their black-box representations. We believe that high model interpretability may help people to break several bottlenecks of deep learning, e.g., learning from very few annotations, learning via human-computer communications at the semantic level, and semantically debugging network representations. We focus on convolutional neural networks (CNNs), and we revisit the visualization of CNN representations, methods of diagnosing representations of pre-trained CNNs, approaches for disentangling pre-trained CNN representations, learning of CNNs with disentangled representations, and middle-to-end learning based on model interpretability. Finally, we discuss prospective trends in explainable artificial intelligence.

Key facts. Cited by 155 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 31; 2022: 58; 2023: 25; 2024: 16; 2025: 14; 2026: 11.

155 papers.

Sort:

By area:

  1. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 299
  2. 2021A Comprehensive Survey on Graph Anomaly Detection with Deep Learningcited by 230
  3. 2021Fine-Grained Image Analysis with Deep Learning: A Surveycited by 135
  4. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
  5. 2021Explainable AI (XAI): A Systematic Meta-Survey of Current Challenges and Future Opportunitiescited by 125
  6. 2022A Comprehensive Survey on Trustworthy Graph Neural Networks: Privacy, Robustness, Fairness, and Explainabilitycited by 118
  7. 2023Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Surveycited by 101
  8. 2021A Comprehensive Taxonomy for Explainable Artificial Intelligence: A Systematic Survey of Surveys on Methods and Conceptscited by 90
  9. 2022Physics-Informed Machine Learning: A Survey on Problems, Methods and Applicationscited by 88
  10. 2021Explainability of deep vision-based autonomous driving systems: Review and challengescited by 80
  11. 2021Explaining Time Series Predictions with Dynamic Maskscited by 72
  12. 2024Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Surveycited by 52
38 more
  1. 2021Exploiting Explanations for Model Inversion Attackscited by 48
  2. 2022Overlooked factors in concept-based explanations: Dataset choice, concept learnability, and human capabilitycited by 46
  3. 2024Usable XAI: 10 Strategies Towards Exploiting Explainability in the LLM Eracited by 45
  4. 2021Explainable Artificial Intelligence Approaches: A Surveycited by 40
  5. 2023Does Physical Adversarial Example Really Matter to Autonomous Driving? Towards System-Level Effect of Adversarial Object Evasion Attackcited by 37
  6. 2021Neural Natural Language Processing for Unstructured Data in Electronic Health Records: a Reviewcited by 35
  7. 2022Going Beyond XAI: A Systematic Survey for Explanation-Guided Learningcited by 35
  8. 2022Guidelines and Evaluation of Clinical Explainable AI in Medical Image Analysiscited by 34
  9. 2021A Little Robustness Goes a Long Way: Leveraging Robust Features for Targeted Transfer Attackscited by 33
  10. 2022DL4SciVis: A State-of-the-Art Survey on Deep Learning for Scientific Visualizationcited by 32
  11. 2022Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectivescited by 32
  12. 2024Backward Lens: Projecting Language Model Gradients into the Vocabulary Spacecited by 30
  13. 2022The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizationscited by 29
  14. 2023Meta-Album: Multi-domain Meta-Dataset for Few-Shot Image Classificationcited by 29
  15. 2021Towards Relatable Explainable AI with the Perceptual Processcited by 27
  16. 2021Fully Transformer Networks for Semantic Image Segmentationcited by 24
  17. 2021Explainable Reinforcement Learning for Broad-XAI: A Conceptual Framework and Surveycited by 24
  18. 2022Causal Reasoning Meets Visual Representation Learning: A Prospective Studycited by 24
  19. 2022Fairness and Explainability: Bridging the Gap Towards Fair Model Explanationscited by 22
  20. 2025SAE-V: Interpreting Multimodal Models for Enhanced Alignmentcited by 22
  21. 2021CX-ToM: Counterfactual Explanations with Theory-of-Mind for Enhancing Human Trust in Image Recognition Modelscited by 20
  22. 2022Concept Embedding Analysis: A Reviewcited by 20
  23. 2024A Survey on Evaluation of Out-of-Distribution Generalizationcited by 20
  24. 2023Explaining Quantum Circuits with Shapley Values: Towards Explainable Quantum Machine Learningcited by 17
  25. 2024Concept-Based Explanations in Computer Vision: Where Are We and Where Could We Go?cited by 17
  26. 2022Attribution-based XAI Methods in Computer Vision: A Reviewcited by 16
  27. 2023Tensor networks for interpretable and efficient quantum-inspired machine learningcited by 16
  28. 2023Interpretation of Time-Series Deep Models: A Surveycited by 15
  29. 2023A Simple Interpretable Transformer for Fine-Grained Image Classification and Analysiscited by 15
  30. 2021A Survey on Understanding, Visualizations, and Explanation of Deep Neural Networkscited by 14
  31. 2021Explainable Activity Recognition for Smart Home Systemscited by 14
  32. 2022Recent Advances in Neural Text Generation: A Task-Agnostic Surveycited by 14
  33. 2023Interpretability of Machine Learning: Recent Advances and Future Prospectscited by 12
  34. 2023FAIRER: Fairness as Decision Rationale Alignmentcited by 12
  35. 2021Toward Open-World Electroencephalogram Decoding Via Deep Learning: A Comprehensive Surveycited by 11
  36. 2022Regression modelling of spatiotemporal extreme U.S. wildfires via partially-interpretable neural networkscited by 11
  37. 2022A Survey of Deep Causal Models and Their Industrial Applicationscited by 10
  38. 2021What do End-to-End Speech Models Learn about Speaker, Language and Channel Information? A Layer-wise and Neuron-level Analysiscited by 9

50 most cited of 155 All 155 (JSON)