# Concept Saliency Maps to Visualize Relevant Features in Deep Generative Models

arXiv: https://arxiv.org/abs/1910.13140 · HTML: https://mlsanity.com/paper/1910.13140/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Oct 2019.

## Built on by (most cited first, 11 of 11)

- [Towards Automated Circuit Discovery for Mechanistic Interpretability](https://mlsanity.com/paper/2304.14997/) (2023)
- [Concept Activation Regions: A Generalized Framework For Concept-Based Explanations](https://mlsanity.com/paper/2209.11222/) (2022)
- [Representation Engineering for Large-Language Models: Survey and Research Challenges](https://mlsanity.com/paper/2502.17601/) (2025)
- [Label-Free Explainability for Unsupervised Models](https://mlsanity.com/paper/2203.01928/) (2022)
- [Class-Discriminative Attention Maps for Vision Transformers](https://mlsanity.com/paper/2312.02364/) (2023)
- [Explaining, Evaluating and Enhancing Neural Networks' Learned Representations](https://mlsanity.com/paper/2202.09374/) (2022)
- [Locally Testing Model Detections for Semantic Global Concepts](https://mlsanity.com/paper/2405.17523/) (2024)
- [Prospector Heads: Generalized Feature Attribution for Large Models & Data](https://mlsanity.com/paper/2402.11729/) (2024)
- [Post-Hoc Concept Disentanglement: From Correlated to Isolated Concept Representations](https://mlsanity.com/paper/2503.05522/) (2025)
- [Probing the Probes: Methods and Metrics for Concept Alignment](https://mlsanity.com/paper/2511.04312/) (2025)
- [Regularizing Attention Scores with Bootstrapping](https://mlsanity.com/paper/2604.01339/) (2026)

## Abstract (arXiv)

Evaluating, explaining, and visualizing high-level concepts in generative models, such as variational autoencoders (VAEs), is challenging in part due to a lack of known prediction classes that are required to generate saliency maps in supervised learning. While saliency maps may help identify relevant features (e.g., pixels) in the input for classification tasks of deep neural networks, similar frameworks are understudied in unsupervised learning. Therefore, we introduce a new method of obtaining saliency maps for latent representations of known or novel high-level concepts, often called concept vectors in generative models. Concept scores, analogous to class scores in classification tasks, are defined as dot products between concept vectors and encoded input data, which can be readily used to compute the gradients. The resulting concept saliency maps are shown to highlight input features deemed important for high-level concepts. Our method is applied to the VAE's latent space of CelebA dataset in which known attributes such as "smiles" and "hats" are used to elucidate relevant facial features. Furthermore, our application to spatial transcriptomic (ST) data of a mouse olfactory bulb demonstrates the potential of latent representations of morphological layers and molecular features in advancing our understanding of complex biological systems. By extending the popular method of saliency maps to generative models, the proposed concept saliency maps help improve interpretability of latent variable models in deep learning. Codes to reproduce and to implement concept saliency maps: https://github.com/lenbrocki/concept-saliency-maps