# ChannelExplorer: Exploring Class Separability Through Activation Channel Visualization

arXiv: https://arxiv.org/abs/2505.04647 · HTML: https://mlsanity.com/paper/2505.04647/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 40 of its 90 references that are in the map.
- First posted on arXiv 6 May 2025.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 40)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [Network Dissection: Quantifying Interpretability of Deep Visual Representations](https://mlsanity.com/paper/1704.05796/) (2017)
- [Deep Domain Confusion: Maximizing for Domain Invariance](https://mlsanity.com/paper/1412.3474/) (2014)
- [Self-Normalizing Neural Networks](https://mlsanity.com/paper/1706.02515/) (2017)
- [Understanding Neural Networks Through Deep Visualization](https://mlsanity.com/paper/1506.06579/) (2015)
- [Net2Vec: Quantifying and Explaining how Concepts are Encoded by Filters in Deep Neural Networks](https://mlsanity.com/paper/1801.03454/) (2018)
- [Hyper-Parameter Optimization: A Review of Algorithms and Applications](https://mlsanity.com/paper/2003.05689/) (2020)
- [Summit: Scaling Deep Learning Interpretability by Visualizing Activation and Attribution Summarizations](https://mlsanity.com/paper/1904.02323/) (2019)
- [Do Convolutional Neural Networks Learn Class Hierarchy?](https://mlsanity.com/paper/1710.06501/) (2017)
- [CNN Explainer: Learning Convolutional Neural Networks with Interactive Visualization](https://mlsanity.com/paper/2004.15004/) (2020)
- [ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Models](https://mlsanity.com/paper/1704.01942/) (2017)
- [GAN Lab: Understanding Complex Deep Generative Models using Interactive Visual Experimentation](https://mlsanity.com/paper/1809.01587/) (2018)
- [Analyzing the Noise Robustness of Deep Neural Networks](https://mlsanity.com/paper/2001.09395/) (2020)
- [Evolution of Image Segmentation using Deep Convolutional Neural Network: A Survey](https://mlsanity.com/paper/2001.04074/) (2020)
- [When Face Recognition Meets with Deep Learning: an Evaluation of Convolutional Neural Networks for Face Recognition](https://mlsanity.com/paper/1504.02351/) (2015)
- [ConceptExplainer: Interactive Explanation for Deep Neural Networks from a Concept Perspective](https://mlsanity.com/paper/2204.01888/) (2022)
- [Neo: Generalizing Confusion Matrix Visualization to Hierarchical and Multi-Output Labels](https://mlsanity.com/paper/2110.12536/) (2021)
- [Diffusion Explainer: Visual Explanation for Text-to-image Stable Diffusion](https://mlsanity.com/paper/2305.03509/) (2023)
- [ConfusionFlow: A model-agnostic visualization for temporal analysis of classifier confusion](https://mlsanity.com/paper/1910.00969/) (2019)
- [Totally Looks Like - How Humans Compare, Compared to Machines](https://mlsanity.com/paper/1803.01485/) (2018)
- [Bluff: Interactively Deciphering Adversarial Attacks on Deep Neural Networks](https://mlsanity.com/paper/2009.02608/) (2020)

## Built on by (most cited first, 1 of 1)

- [HOLE: Homological Observation of Latent Embeddings for Neural Network Interpretability](https://mlsanity.com/paper/2512.07988/) (2025)

## Abstract (arXiv)

Deep neural networks (DNNs) achieve state-of-the-art performance in many vision tasks, yet understanding their internal behavior remains challenging, particularly how different layers and activation channels contribute to class separability. We introduce ChannelExplorer, an interactive visual analytics tool for analyzing image-based outputs across model layers, emphasizing data-driven insights over architecture analysis for exploring class separability. ChannelExplorer summarizes activations across layers and visualizes them using three primary coordinated views: a Scatterplot View to reveal inter- and intra-class confusion, a Jaccard Similarity View to quantify activation overlap, and a Heatmap View to inspect activation channel patterns. Our technique supports diverse model architectures, including CNNs, GANs, ResNet and Stable Diffusion models. We demonstrate the capabilities of ChannelExplorer through four use-case scenarios: (1) generating class hierarchy in ImageNet, (2) finding mislabeled images, (3) identifying activation channel contributions, and(4) locating latent states' position in Stable Diffusion model. Finally, we evaluate the tool with expert users.