# ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Models

arXiv: https://arxiv.org/abs/1704.01942 · HTML: https://mlsanity.com/paper/1704.01942/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Apr 2017.

## Built on by (most cited first, 30 of 56)

- [Exploiting Explanations for Model Inversion Attacks](https://mlsanity.com/paper/2104.12669/) (2021)
- [M2Lens: Visualizing and Explaining Multimodal Models for Sentiment Analysis](https://mlsanity.com/paper/2107.08264/) (2021)
- [Shared Interest: Measuring Human-AI Alignment to Identify Recurring Patterns in Model Behavior](https://mlsanity.com/paper/2107.09234/) (2021)
- [The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizations](https://mlsanity.com/paper/2212.11737/) (2022)
- [LLM Comparator: Visual Analytics for Side-by-Side Evaluation of Large Language Models](https://mlsanity.com/paper/2402.10524/) (2024)
- [Aligning Eyes between Humans and Deep Neural Network through Interactive Attention Alignment](https://mlsanity.com/paper/2202.02838/) (2022)
- [Taxonomy of Machine Learning Safety: A Survey and Primer](https://mlsanity.com/paper/2106.04823/) (2021)
- [Visual Analytics For Machine Learning: A Data Perspective Survey](https://mlsanity.com/paper/2307.07712/) (2023)
- [Sibyl: Understanding and Addressing the Usability Challenges of Machine Learning In High-Stakes Decision Making](https://mlsanity.com/paper/2103.02071/) (2021)
- [Symphony: Composing Interactive Interfaces for Machine Learning](https://mlsanity.com/paper/2202.08946/) (2022)
- [Neo: Generalizing Confusion Matrix Visualization to Hierarchical and Multi-Output Labels](https://mlsanity.com/paper/2110.12536/) (2021)
- [Diffusion Explainer: Visual Explanation for Text-to-image Stable Diffusion](https://mlsanity.com/paper/2305.03509/) (2023)
- [Diagnosing Ensemble Few-Shot Classifiers](https://mlsanity.com/paper/2206.04372/) (2022)
- [Model LineUpper: Supporting Interactive Model Comparison at Multiple Levels for AutoML](https://mlsanity.com/paper/2104.04375/) (2021)
- [DendroMap: Visual Exploration of Large-Scale Image Datasets for Machine Learning with Treemaps](https://mlsanity.com/paper/2205.06935/) (2022)
- [T3-Vis: a visual analytic framework for Training and fine-Tuning Transformers in NLP](https://mlsanity.com/paper/2108.13587/) (2021)
- [Learning-From-Disagreement: A Model Comparison and Visual Analytics Framework](https://mlsanity.com/paper/2201.07849/) (2022)
- [GAM Coach: Towards Interactive and User-centered Algorithmic Recourse](https://mlsanity.com/paper/2302.14165/) (2023)
- [CLAMS: A Cluster Ambiguity Measure for Estimating Perceptual Variability in Visual Clustering](https://mlsanity.com/paper/2308.00284/) (2023)
- [Stop Misusing t-SNE and UMAP for Visual Analytics](https://mlsanity.com/paper/2506.08725/) (2025)
- [Neural Activation Patterns (NAPs): Visual Explainability of Learned Concepts](https://mlsanity.com/paper/2206.10611/) (2022)
- [Explainable Adversarial Attacks in Deep Neural Networks Using Activation Profiles](https://mlsanity.com/paper/2103.10229/) (2021)
- [Calibrate: Interactive Analysis of Probabilistic Model Output](https://mlsanity.com/paper/2207.13770/) (2022)
- [A Visual Analytics System for Improving Attention-based Traffic Forecasting Models](https://mlsanity.com/paper/2208.04350/) (2022)
- [DeepSeer: Interactive RNN Explanation and Debugging via State Abstraction](https://mlsanity.com/paper/2303.01576/) (2023)
- [ProtoExplorer: Interpretable Forensic Analysis of Deepfake Videos using Prototype Exploration and Refinement](https://mlsanity.com/paper/2309.11155/) (2023)
- [Allowing humans to interactively guide machines where to look does not always improve human-AI team's classification accuracy](https://mlsanity.com/paper/2404.05238/) (2024)
- [Visualizing and Explaining Language Models](https://mlsanity.com/paper/2205.10238/) (2022)
- [Compress and Compare: Interactively Evaluating Efficiency and Behavior Across ML Model Compression Experiments](https://mlsanity.com/paper/2408.03274/) (2024)
- [Unlearning Comparator: A Visual Analytics System for Comparative Evaluation of Machine Unlearning Methods](https://mlsanity.com/paper/2508.12730/) (2025)

## Abstract (arXiv)

While deep learning models have achieved state-of-the-art accuracies for many prediction tasks, understanding these models remains a challenge. Despite the recent interest in developing visual tools to help users interpret deep learning models, the complexity and wide variety of models deployed in industry, and the large-scale datasets that they used, pose unique design challenges that are inadequately addressed by existing work. Through participatory design sessions with over 15 researchers and engineers at Facebook, we have developed, deployed, and iteratively improved ActiVis, an interactive visualization system for interpreting large-scale deep learning models and results. By tightly integrating multiple coordinated views, such as a computation graph overview of the model architecture, and a neuron activation view for pattern discovery and comparison, users can explore complex deep neural network models at both the instance- and subset-level. ActiVis has been deployed on Facebook's machine learning platform. We present case studies with Facebook researchers and engineers, and usage scenarios of how ActiVis may work with different models.