# Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networks

arXiv: https://arxiv.org/abs/1602.03616 · HTML: https://mlsanity.com/paper/1602.03616/

## Key facts

- Cited by 106 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Feb 2016.

## Built on by (most cited first, 30 of 106)

- [Finding Neurons in a Haystack: Case Studies with Sparse Probing](https://mlsanity.com/paper/2305.01610/) (2023)
- [Reconstructing Training Data from Trained Neural Networks](https://mlsanity.com/paper/2206.07758/) (2022)
- [Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks](https://mlsanity.com/paper/2207.13243/) (2022)
- [Salient ImageNet: How to discover spurious features in Deep Learning?](https://mlsanity.com/paper/2110.04301/) (2021)
- [Universal Neurons in GPT2 Language Models](https://mlsanity.com/paper/2401.12181/) (2024)
- [A Comprehensive Study of Image Classification Model Sensitivity to Foregrounds, Backgrounds, and Visual Attributes](https://mlsanity.com/paper/2201.10766/) (2022)
- [Mathematical Models of Computation in Superposition](https://mlsanity.com/paper/2408.05451/) (2024)
- [Interpreting Neural Networks through the Polytope Lens](https://mlsanity.com/paper/2211.12312/) (2022)
- [Human-Centric Multimodal Machine Learning: Recent Advances and Testbed on AI-based Recruitment](https://mlsanity.com/paper/2302.10908/) (2023)
- [ImageNet-Hard: The Hardest Images Remaining from a Study of the Power of Zoom and Spatial Biases in Image Classification](https://mlsanity.com/paper/2304.05538/) (2023)
- [Don't trust your eyes: on the (un)reliability of feature visualizations](https://mlsanity.com/paper/2306.04719/) (2023)
- [PURE: Turning Polysemantic Neurons Into Pure Features by Identifying Relevant Circuits](https://mlsanity.com/paper/2404.06453/) (2024)
- [Disentangling Neuron Representations with Concept Vectors](https://mlsanity.com/paper/2304.09707/) (2023)
- [Unlocking Feature Visualization for Deeper Networks with MAgnitude Constrained Optimization](https://mlsanity.com/paper/2306.06805/) (2023)
- [Enhancing Neural Network Interpretability with Feature-Aligned Sparse Autoencoders](https://mlsanity.com/paper/2411.01220/) (2024)
- [Xplique: A Deep Learning Explainability Toolbox](https://mlsanity.com/paper/2206.04394/) (2022)
- [Scientific Inference With Interpretable Machine Learning: Analyzing Models to Learn About Real-World Phenomena](https://mlsanity.com/paper/2206.05487/) (2022)
- [Explaining Bayesian Neural Networks](https://mlsanity.com/paper/2108.10346/) (2021)
- [DeepAID: Interpreting and Improving Deep Learning-based Anomaly Detection in Security Applications](https://mlsanity.com/paper/2109.11495/) (2021)
- [Core Risk Minimization using Salient ImageNet](https://mlsanity.com/paper/2203.15566/) (2022)
- [Towards a fuller understanding of neurons with Clustered Compositional Explanations](https://mlsanity.com/paper/2310.18443/) (2023)
- [Understanding Visual Feature Reliance through the Lens of Complexity](https://mlsanity.com/paper/2407.06076/) (2024)
- [A Survey on Understanding, Visualizations, and Explanation of Deep Neural Networks](https://mlsanity.com/paper/2102.01792/) (2021)
- [DORA: Exploring Outlier Representations in Deep Neural Networks](https://mlsanity.com/paper/2206.04530/) (2022)
- [From Black Boxes to Conversations: Incorporating XAI in a Conversational Agent](https://mlsanity.com/paper/2209.02552/) (2022)
- [Ten Years of Generative Adversarial Nets (GANs): A survey of the state-of-the-art](https://mlsanity.com/paper/2308.16316/) (2023)
- [Using Degeneracy in the Loss Landscape for Mechanistic Interpretability](https://mlsanity.com/paper/2405.10927/) (2024)
- [Beyond Interpretability: The Gains of Feature Monosemanticity on Model Robustness](https://mlsanity.com/paper/2410.21331/) (2024)
- [NeuroCartography: Scalable Automatic Visual Summarization of Concepts in Deep Neural Networks](https://mlsanity.com/paper/2108.12931/) (2021)
- [IFBiD: Inference-Free Bias Detection](https://mlsanity.com/paper/2109.04374/) (2021)

## Abstract (arXiv)

We can better understand deep neural networks by identifying which features each of their neurons have learned to detect. To do so, researchers have created Deep Visualization techniques including activation maximization, which synthetically generates inputs (e.g. images) that maximally activate each neuron. A limitation of current techniques is that they assume each neuron detects only one type of feature, but we know that neurons can be multifaceted, in that they fire in response to many different types of features: for example, a grocery store class neuron must activate either for rows of produce or for a storefront. Previous activation maximization techniques constructed images without regard for the multiple different facets of a neuron, creating inappropriate mixes of colors, parts of objects, scales, orientations, etc. Here, we introduce an algorithm that explicitly uncovers the multiple facets of each neuron by producing a synthetic visualization of each of the types of images that activate a neuron. We also introduce regularization methods that produce state-of-the-art results in terms of the interpretability of images obtained by activation maximization. By separately synthesizing each type of image a neuron fires in response to, the visualizations have more appropriate colors and coherent global structure. Multifaceted feature visualization thus provides a clearer and more comprehensive description of the role of each neuron.