# TopoAct: Visually Exploring the Shape of Activations in Deep Learning

arXiv: https://arxiv.org/abs/1912.06332 · HTML: https://mlsanity.com/paper/1912.06332/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Dec 2019.

## Built on by (most cited first, 4 of 4)

- [Explainable Mapper: Charting LLM Embedding Spaces Using Perturbation-Based Explanation and Verification Agents](https://mlsanity.com/paper/2507.18607/) (2025)
- [VERB: Visualizing and Interpreting Bias Mitigation Techniques for Word Representations](https://mlsanity.com/paper/2104.02797/) (2021)
- [Mixup Barcodes: Quantifying Geometric-Topological Interactions between Point Clouds](https://mlsanity.com/paper/2402.15058/) (2024)
- [Visual Exploration of Feature Relationships in Sparse Autoencoders with Curated Concepts](https://mlsanity.com/paper/2511.06048/) (2025)

## Abstract (arXiv)

Deep neural networks such as GoogLeNet, ResNet, and BERT have achieved impressive performance in tasks such as image and text classification. To understand how such performance is achieved, we probe a trained deep neural network by studying neuron activations, i.e., combinations of neuron firings, at various layers of the network in response to a particular input. With a large number of inputs, we aim to obtain a global view of what neurons detect by studying their activations. In particular, we develop visualizations that show the shape of the activation space, the organizational principle behind neuron activations, and the relationships of these activations within a layer. Applying tools from topological data analysis, we present TopoAct, a visual exploration system to study topological summaries of activation vectors. We present exploration scenarios using TopoAct that provide valuable insights into learned representations of neural networks. We expect TopoAct to give a topological perspective that enriches the current toolbox of neural network analysis, and to provide a basis for network architecture diagnosis and data anomaly detection.