# Understanding Neural Networks via Feature Visualization: A survey

arXiv: https://arxiv.org/abs/1904.08939 · HTML: https://mlsanity.com/paper/1904.08939/

## Key facts

- Cited by 65 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Apr 2019.

## Built on by (most cited first, 30 of 65)

- [Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks](https://mlsanity.com/paper/2207.13243/) (2022)
- [A Comprehensive Taxonomy for Explainable Artificial Intelligence: A Systematic Survey of Surveys on Methods and Concepts](https://mlsanity.com/paper/2105.07190/) (2021)
- [The effectiveness of feature attribution methods and its correlation with automatic evaluation scores](https://mlsanity.com/paper/2105.14944/) (2021)
- [Reliability of CKA as a Similarity Measure in Deep Learning](https://mlsanity.com/paper/2210.16156/) (2022)
- [A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Models](https://mlsanity.com/paper/2503.05613/) (2025)
- [Universal Sparse Autoencoders: Interpretable Cross-Model Concept Alignment](https://mlsanity.com/paper/2502.03714/) (2025)
- [Don't trust your eyes: on the (un)reliability of feature visualizations](https://mlsanity.com/paper/2306.04719/) (2023)
- [Distilling Robust and Non-Robust Features in Adversarial Examples by Information Bottleneck](https://mlsanity.com/paper/2204.02735/) (2022)
- [How Well do Feature Visualizations Support Causal Understanding of CNN Activations?](https://mlsanity.com/paper/2106.12447/) (2021)
- [Xplique: A Deep Learning Explainability Toolbox](https://mlsanity.com/paper/2206.04394/) (2022)
- [Understanding the (Extra-)Ordinary: Validating Deep Model Decisions with Prototypical Concept-based Explanations](https://mlsanity.com/paper/2311.16681/) (2023)
- [Dimensions underlying the representational alignment of deep neural networks with humans](https://mlsanity.com/paper/2406.19087/) (2024)
- [Masking Adversarial Damage: Finding Adversarial Saliency for Robust and Sparse Network](https://mlsanity.com/paper/2204.02738/) (2022)
- [Concept Embedding Analysis: A Review](https://mlsanity.com/paper/2203.13909/) (2022)
- [Surrogate Model-Based Explainability Methods for Point Cloud NNs](https://mlsanity.com/paper/2107.13459/) (2021)
- [Demystifying Causal Features on Adversarial Examples and Causal Inoculation for Robust Network by Adversarial Instrumental Variable Regression](https://mlsanity.com/paper/2303.01052/) (2023)
- [Disruptive Autoencoders: Leveraging Low-level features for 3D Medical Image Pre-training](https://mlsanity.com/paper/2307.16896/) (2023)
- [Understanding Visual Feature Reliance through the Lens of Complexity](https://mlsanity.com/paper/2407.06076/) (2024)
- [DORA: Exploring Outlier Representations in Deep Neural Networks](https://mlsanity.com/paper/2206.04530/) (2022)
- [LLMScan: Causal Scan for LLM Misbehavior Detection](https://mlsanity.com/paper/2410.16638/) (2024)
- [Explainability of Vision Transformers: A Comprehensive Review and New Perspectives](https://mlsanity.com/paper/2311.06786/) (2023)
- [EIVEN: Efficient Implicit Attribute Value Extraction using Multimodal LLM](https://mlsanity.com/paper/2404.08886/) (2024)
- [Understanding Reasoning in Chain-of-Thought from the Hopfieldian View](https://mlsanity.com/paper/2410.03595/) (2024)
- [Revisiting the Performance-Explainability Trade-Off in Explainable Artificial Intelligence (XAI)](https://mlsanity.com/paper/2307.14239/) (2023)
- [Preemptively Pruning Clever-Hans Strategies in Deep Neural Networks](https://mlsanity.com/paper/2304.05727/) (2023)
- [Identifying Interpretable Visual Features in Artificial and Biological Neural Systems](https://mlsanity.com/paper/2310.11431/) (2023)
- [Visualizing Global Explanations of Point Cloud DNNs](https://mlsanity.com/paper/2203.09505/) (2022)
- [Explainability-Aware One Point Attack for Point Cloud Neural Networks](https://mlsanity.com/paper/2110.04158/) (2021)
- [Inverting Adversarially Robust Networks for Image Synthesis](https://mlsanity.com/paper/2106.06927/) (2021)
- [Explaining, Evaluating and Enhancing Neural Networks' Learned Representations](https://mlsanity.com/paper/2202.09374/) (2022)

## Abstract (arXiv)

A neuroscience method to understanding the brain is to find and study the preferred stimuli that highly activate an individual cell or groups of cells. Recent advances in machine learning enable a family of methods to synthesize preferred stimuli that cause a neuron in an artificial or biological brain to fire strongly. Those methods are known as Activation Maximization (AM) or Feature Visualization via Optimization. In this chapter, we (1) review existing AM techniques in the literature; (2) discuss a probabilistic interpretation for AM; and (3) review the applications of AM in debugging and explaining networks.