# ConceptExplainer: Interactive Explanation for Deep Neural Networks from a Concept Perspective

arXiv: https://arxiv.org/abs/2204.01888 · HTML: https://mlsanity.com/paper/2204.01888/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Apr 2022.

## Built on by (most cited first, 18 of 18)

- [Visual Analytics For Machine Learning: A Data Perspective Survey](https://mlsanity.com/paper/2307.07712/) (2023)
- [BiasTestGPT: Using ChatGPT for Social Bias Testing of Language Models](https://mlsanity.com/paper/2302.07371/) (2023)
- [Evaluating the Effectiveness of XAI Techniques for Encoder-Based Language Models](https://mlsanity.com/paper/2501.15374/) (2025)
- [A Unified Framework with Novel Metrics for Evaluating the Effectiveness of XAI Techniques in LLMs](https://mlsanity.com/paper/2503.05050/) (2025)
- [AttributionScanner: A Visual Analytics System for Model Validation with Metadata-Free Slice Finding](https://mlsanity.com/paper/2401.06462/) (2024)
- [Finspector: A Human-Centered Visual Inspection Tool for Exploring and Comparing Biases among Foundation Models](https://mlsanity.com/paper/2305.16937/) (2023)
- [ASAP: Interpretable Analysis and Summarization of AI-generated Image Patterns at Scale](https://mlsanity.com/paper/2404.02990/) (2024)
- [Unlearning Comparator: A Visual Analytics System for Comparative Evaluation of Machine Unlearning Methods](https://mlsanity.com/paper/2508.12730/) (2025)
- [Are We Closing the Loop Yet? Gaps in the Generalizability of VIS4ML Research](https://mlsanity.com/paper/2308.06290/) (2023)
- [Latent Space Explorer: Visual Analytics for Multimodal Latent Space Exploration](https://mlsanity.com/paper/2312.00857/) (2023)
- [InterVLS: Interactive Model Understanding and Improvement with Vision-Language Surrogates](https://mlsanity.com/paper/2311.03547/) (2023)
- [MiMICRI: Towards Domain-centered Counterfactual Explanations of Cardiovascular Image Classification Models](https://mlsanity.com/paper/2404.16174/) (2024)
- [ChannelExplorer: Exploring Class Separability Through Activation Channel Visualization](https://mlsanity.com/paper/2505.04647/) (2025)
- [ConceptViz: A Visual Analytics Approach for Exploring Concepts in Large Language Models](https://mlsanity.com/paper/2509.20376/) (2025)
- [LINGO : Visually Debiasing Natural Language Instructions to Support Task Diversity](https://mlsanity.com/paper/2304.06184/) (2023)
- [InFiConD: Interactive No-code Fine-tuning with Concept-based Knowledge Distillation](https://mlsanity.com/paper/2406.17838/) (2024)
- [Explanation User Interfaces: A Systematic Literature Review](https://mlsanity.com/paper/2505.20085/) (2025)
- [ASAP: Visual Analytics for Identifying and Analyzing Image Patterns in AI-generated Images](https://mlsanity.com/paper/2609.27371/) (2026)

## Abstract (arXiv)

Traditional deep learning interpretability methods which are suitable for model users cannot explain network behaviors at the global level and are inflexible at providing fine-grained explanations. As a solution, concept-based explanations are gaining attention due to their human intuitiveness and their flexibility to describe both global and local model behaviors. Concepts are groups of similarly meaningful pixels that express a notion, embedded within the network's latent space and have commonly been hand-generated, but have recently been discovered by automated approaches. Unfortunately, the magnitude and diversity of discovered concepts makes it difficult to navigate and make sense of the concept space. Visual analytics can serve a valuable role in bridging these gaps by enabling structured navigation and exploration of the concept space to provide concept-based insights of model behavior to users. To this end, we design, develop, and validate ConceptExplainer, a visual analytics system that enables people to interactively probe and explore the concept space to explain model behavior at the instance/class/global level. The system was developed via iterative prototyping to address a number of design challenges that model users face in interpreting the behavior of deep learning models. Via a rigorous user study, we validate how ConceptExplainer supports these challenges. Likewise, we conduct a series of usage scenarios to demonstrate how the system supports the interactive analysis of model behavior across a variety of tasks and explanation granularities, such as identifying concepts that are important to classification, identifying bias in training data, and understanding how concepts can be shared across diverse and seemingly dissimilar classes.