mlsanityData to 7 Oct 2026

Understanding Neural Networks Through Deep Visualization

arXiv 1506.06579 ↗First posted 22 Jun 2015cs.CV cs.LG cs.NEcited by 332

Not on the map. Why?

Abstract (arXiv)

Recent years have produced great advances in training large, deep neural networks (DNNs), including notable successes in training convolutional neural networks (convnets) to recognize natural images. However, our understanding of how these models work, especially what computations they perform at intermediate layers, has lagged behind. Progress in the field will be further accelerated by the development of better tools for visualizing and interpreting neural nets. We introduce two such tools here. The first is a tool that visualizes the activations produced on each layer of a trained convnet as it processes an image or video (e.g. a live webcam stream). We have found that looking at live activations that change in response to user input helps build valuable intuitions about how convnets work. The second tool enables visualizing features at each layer of a DNN via regularized optimization in image space. Because previous versions of this idea produced less recognizable images, here we introduce several new regularization methods that combine to produce qualitatively clearer, more interpretable visualizations. Both tools are open source and work on a pre-trained convnet with minimal setup.

Key facts. Cited by 332 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Jun 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 83; 2022: 71; 2023: 68; 2024: 61; 2025: 36; 2026: 13.

332 papers.

Sort:

By area:

  1. 2024Do Llamas Work in English? On the Latent Language of Multilingual Transformerscited by 275
  2. 2021BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG datacited by 194
  3. 2023Mitigating the Alignment Tax of RLHFcited by 181
  4. 2022Reconstructing Training Data from Trained Neural Networkscited by 129
  5. 2021Deformable ProtoPNet: An Interpretable Image Classifier Using Deformable Prototypescited by 120
  6. 2021Salient ImageNet: How to discover spurious features in Deep Learning?cited by 114
  7. 2024Rethinking Interpretability in the Era of Large Language Modelscited by 110
  8. 2021Federated Unlearning via Class-Discriminative Pruningcited by 108
  9. 2024Merging Multi-Task Models via Weight-Ensembling Mixture of Expertscited by 85
  10. 2021Leveraging Sparse Linear Layers for Debuggable Deep Networkscited by 82
  11. 2022Towards Adversarial Evaluations for Inexact Machine Unlearningcited by 82
  12. 2024Marabou 2.0: A Versatile Formal Analyzer of Neural Networkscited by 81
38 more
  1. 2021Intriguing Findings of Frequency Selection for Image Deblurringcited by 80
  2. 2022On the Symmetries of Deep Learning Models and their Internal Representationscited by 63
  3. 2022Visual Recognition with Deep Nearest Centroidscited by 59
  4. 2022A Review of Deep Transfer Learning and Recent Advancementscited by 56
  5. 2021The Who in XAI: How AI Background Shapes Perceptions of AI Explanationscited by 48
  6. 2023Mitigating Spurious Correlations in Multi-modal Models during Fine-tuningcited by 47
  7. 2021Domain Adversarial Neural Networks for Domain Generalization: When It Works and How to Improvecited by 46
  8. 2022Margin-Based Few-Shot Class-Incremental Learning with Class-Level Overfitting Mitigationcited by 45
  9. 2021This looks more like that: Enhancing Self-Explaining Models by Prototypical Relevance Propagationcited by 43
  10. 2021Fed2: Feature-Aligned Federated Learningcited by 42
  11. 2021Automatic Sleep Staging of EEG Signals: Recent Development, Challenges, and Future Directionscited by 41
  12. 2022One Model to Synthesize Them All: Multi-contrast Multi-scale Transformer for Missing Data Imputationcited by 40
  13. 2021Biometrics: Trust, but Verifycited by 39
  14. 2022Model Zoos: A Dataset of Diverse Populations of Neural Network Modelscited by 38
  15. 2023Attention-based Point Cloud Edge Samplingcited by 37
  16. 2021Defining and Quantifying the Emergence of Sparse Concepts in DNNscited by 36
  17. 2022MultiViz: Towards Visualizing and Understanding Multimodal Modelscited by 36
  18. 2023This Looks Like Those: Illuminating Prototypical Concepts Using Multiple Visualizationscited by 36
  19. 2022DIME: Fine-grained Interpretations of Multimodal Models via Disentangled Local Explanationscited by 34
  20. 2022Explaining Deepfake Detection by Analysing Image Matchingcited by 34
  21. 2022ProtoVAE: A Trustworthy Self-Explainable Prototypical Variational Modelcited by 34
  22. 2023Charting the Sociotechnical Gap in Explainable AI: A Framework to Address the Gap in XAIcited by 33
  23. 2023Human-Centric Multimodal Machine Learning: Recent Advances and Testbed on AI-based Recruitmentcited by 32
  24. 2023Don't trust your eyes: on the (un)reliability of feature visualizationscited by 32
  25. 2023Focus the Discrepancy: Intra- and Inter-Correlation Learning for Image Anomaly Detectioncited by 32
  26. 2023SplineCam: Exact Visualization and Characterization of Deep Network Geometry and Decision Boundariescited by 31
  27. 2022Distilling Robust and Non-Robust Features in Adversarial Examples by Information Bottleneckcited by 30
  28. 2022Block Selection Method for Using Feature Norm in Out-of-distribution Detectioncited by 30
  29. 2023Towards Trustworthy and Aligned Machine Learning: A Data-centric Survey with Causality Perspectivescited by 30
  30. 2022The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizationscited by 29
  31. 2023Does a Neural Network Really Encode Symbolic Concepts?cited by 27
  32. 2021IMAGINE: Image Synthesis by Image-Guided Model Inversioncited by 25
  33. 2023Opti-CAM: Optimizing saliency maps for interpretabilitycited by 25
  34. 2022Universal Adversarial Examples in Remote Sensing: Methodology and Benchmarkcited by 24
  35. 2022Investigating the Properties of Neural Network Representations in Reinforcement Learningcited by 24
  36. 2024Decoding Probing: Revealing Internal Linguistic Structures in Neural Language Models using Minimal Pairscited by 24
  37. 2021Carrying out CNN Channel Pruning in a White Boxcited by 22
  38. 2023Controllable Inversion of Black-Box Face Recognition Models via Diffusioncited by 22

50 most cited of 332 All 332 (JSON)