mlsanityData to 7 Oct 2026

Understanding Neural Networks via Feature Visualization: A survey

arXiv 1904.08939 ↗First posted 18 Apr 2019cs.LG cs.AI cs.CVcited by 65

Not on the map. Why?

Abstract (arXiv)

A neuroscience method to understanding the brain is to find and study the preferred stimuli that highly activate an individual cell or groups of cells. Recent advances in machine learning enable a family of methods to synthesize preferred stimuli that cause a neuron in an artificial or biological brain to fire strongly. Those methods are known as Activation Maximization (AM) or Feature Visualization via Optimization. In this chapter, we (1) review existing AM techniques in the literature; (2) discuss a probabilistic interpretation for AM; and (3) review the applications of AM in debugging and explaining networks.

Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 16; 2023: 17; 2024: 10; 2025: 8; 2026: 3.

65 papers.

Sort:

By area:

  1. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
  2. 2021A Comprehensive Taxonomy for Explainable Artificial Intelligence: A Systematic Survey of Surveys on Methods and Conceptscited by 90
  3. 2021The effectiveness of feature attribution methods and its correlation with automatic evaluation scorescited by 87
  4. 2022Reliability of CKA as a Similarity Measure in Deep Learningcited by 82
  5. 2025A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Modelscited by 73
  6. 2025Universal Sparse Autoencoders: Interpretable Cross-Model Concept Alignmentcited by 62
  7. 2023Don't trust your eyes: on the (un)reliability of feature visualizationscited by 32
  8. 2022Distilling Robust and Non-Robust Features in Adversarial Examples by Information Bottleneckcited by 30
  9. 2021How Well do Feature Visualizations Support Causal Understanding of CNN Activations?cited by 28
  10. 2022Xplique: A Deep Learning Explainability Toolboxcited by 24
  11. 2024Dimensions underlying the representational alignment of deep neural networks with humanscited by 22
  12. 2022Masking Adversarial Damage: Finding Adversarial Saliency for Robust and Sparse Networkcited by 21
38 more
  1. 2023Understanding the (Extra-)Ordinary: Validating Deep Model Decisions with Prototypical Concept-based Explanationscited by 21
  2. 2022Concept Embedding Analysis: A Reviewcited by 20
  3. 2021Surrogate Model-Based Explainability Methods for Point Cloud NNscited by 18
  4. 2023Demystifying Causal Features on Adversarial Examples and Causal Inoculation for Robust Network by Adversarial Instrumental Variable Regressioncited by 18
  5. 2023Disruptive Autoencoders: Leveraging Low-level features for 3D Medical Image Pre-trainingcited by 15
  6. 2024Understanding Visual Feature Reliance through the Lens of Complexitycited by 15
  7. 2022DORA: Exploring Outlier Representations in Deep Neural Networkscited by 14
  8. 2024LLMScan: Causal Scan for LLM Misbehavior Detectioncited by 13
  9. 2023Explainability of Vision Transformers: A Comprehensive Review and New Perspectivescited by 12
  10. 2024EIVEN: Efficient Implicit Attribute Value Extraction using Multimodal LLMcited by 12
  11. 2023Revisiting the Performance-Explainability Trade-Off in Explainable Artificial Intelligence (XAI)cited by 10
  12. 2024Understanding Reasoning in Chain-of-Thought from the Hopfieldian Viewcited by 10
  13. 2023Preemptively Pruning Clever-Hans Strategies in Deep Neural Networkscited by 9
  14. 2023Identifying Interpretable Visual Features in Artificial and Biological Neural Systemscited by 9
  15. 2022Visualizing Global Explanations of Point Cloud DNNscited by 8
  16. 2021Explainability-Aware One Point Attack for Point Cloud Neural Networkscited by 7
  17. 2021Inverting Adversarially Robust Networks for Image Synthesiscited by 6
  18. 2022Explaining, Evaluating and Enhancing Neural Networks' Learned Representationscited by 6
  19. 2022Towards Human-Compatible XAI: Explaining Data Differentials with Concept Induction over Background Knowledgecited by 6
  20. 2023Explainable AI applications in the Medical Domain: a systematic reviewcited by 5
  21. 2022Visualizing and Explaining Language Modelscited by 4
  22. 2022Towards Human-Interpretable Prototypes for Visual Assessment of Image Classification Modelscited by 4
  23. 2023Revealing Similar Semantics Inside CNNs: An Interpretable Concept-based Comparison of Feature Spacescited by 4
  24. 2025AlignSAE: Concept-Aligned Sparse Autoencoderscited by 4
  25. 2022Explaining the root causes of unit-level changescited by 3
  26. 2022On the Evaluation of the Plausibility and Faithfulness of Sentiment Analysis Explanationscited by 3
  27. 2023Causal Temporal Graph Convolutional Neural Networks (CTGCN)cited by 3
  28. 2023Neuron Sensitivity Guided Test Case Selection for Deep Learning Testingcited by 3
  29. 2023Investigating the Encoding of Words in BERT's Neurons using Feature Textualizationcited by 3
  30. 2024Manipulating Feature Visualizations with Gradient Slingshotscited by 3
  31. 2026Structuring Sparsity: Block-Sparse Featurizers Capture Visual Concept Manifoldscited by 3
  32. 2023Trainable and Explainable Simplicial Map Neural Networkscited by 2
  33. 2024CoProNN: Concept-based Prototypical Nearest Neighbors for Explaining Vision Modelscited by 2
  34. 2024Leaky ReLUs That Differ in Forward and Backward Pass Facilitate Activation Maximization in Deep Neural Networkscited by 2
  35. 2021Picasso: Model-free Feature Visualizationcited by 1
  36. 2021Disentangled representations: towards interpretation of sex determination from hip bonecited by 1
  37. 2023Applying language models to algebraic topology: generating simplicial cycles using multi-labeling in Wu's formulacited by 1
  38. 2024Interpreting Global Perturbation Robustness of Image Models using Axiomatic Spectral Importance Decompositioncited by 1

50 most cited of 65 All 65 (JSON)