# Experimental Observations of the Topology of Convolutional Neural Network Activations

arXiv: https://arxiv.org/abs/2212.00222 · HTML: https://mlsanity.com/paper/2212.00222/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 37 references that are in the map.
- First posted on arXiv 1 Dec 2022.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Understanding Deep Image Representations by Inverting Them](https://mlsanity.com/paper/1412.0035/) (2014)
- [Insights on representational similarity in neural networks with canonical correlation](https://mlsanity.com/paper/1806.05759/) (2018)
- [Object Detectors Emerge in Deep Scene CNNs](https://mlsanity.com/paper/1412.6856/) (2014)
- [Sliced Wasserstein Kernel for Persistence Diagrams](https://mlsanity.com/paper/1706.03358/) (2017)
- [Neural Persistence: A Complexity Measure for Deep Neural Networks Using Algebraic Topology](https://mlsanity.com/paper/1812.09764/) (2018)
- [Representation Topology Divergence: A Method for Comparing Neural Network Representations](https://mlsanity.com/paper/2201.00058/) (2021)
- [On Characterizing the Capacity of Neural Networks using Algebraic Topology](https://mlsanity.com/paper/1802.04443/) (2018)
- [Understanding Intra-Class Knowledge Inside CNN](https://mlsanity.com/paper/1507.02379/) (2015)
- [Exposition and Interpretation of the Topology of Neural Networks](https://mlsanity.com/paper/1810.03234/) (2018)
- [Topological Uncertainty: Monitoring trained neural networks through persistence of activation graphs](https://mlsanity.com/paper/2105.04404/) (2021)
- [Activation Landscapes as a Topological Summary of Neural Network Performance](https://mlsanity.com/paper/2110.10136/) (2021)
- [Mapper Interactive: A Scalable, Extendable, and Interactive Toolbox for the Visual Exploration of High-Dimensional Data](https://mlsanity.com/paper/2011.03209/) (2020)

## Built on by (most cited first, 10 of 10)

- [Position: Topological Deep Learning is the New Frontier for Relational Learning](https://mlsanity.com/paper/2402.08871/) (2024)
- [Mapping the Multiverse of Latent Representations](https://mlsanity.com/paper/2402.01514/) (2024)
- [Explainable Mapper: Charting LLM Embedding Spaces Using Perturbation-Based Explanation and Verification Agents](https://mlsanity.com/paper/2507.18607/) (2025)
- [Semantic Depth Matters: Explaining Errors of Deep Vision Networks through Perceived Class Similarities](https://mlsanity.com/paper/2504.09956/) (2025)
- [Topological Invariance and Breakdown in Learning](https://mlsanity.com/paper/2510.02670/) (2025)
- [HOLE: Homological Observation of Latent Embeddings for Neural Network Interpretability](https://mlsanity.com/paper/2512.07988/) (2025)
- [Teaching People LLM's Errors and Getting it Right](https://mlsanity.com/paper/2512.21422/) (2025)
- [Searching for internal symbols underlying deep learning](https://mlsanity.com/paper/2405.20605/) (2024)
- [TopoAlign: Topology-Aware Visual Representation Alignment](https://mlsanity.com/paper/2605.25541/) (2026)
- [TopoTuner: Topological Finetuning of Large Language Models](https://mlsanity.com/paper/2607.16637/) (2026)

## Abstract (arXiv)

Topological data analysis (TDA) is a branch of computational mathematics, bridging algebraic topology and data science, that provides compact, noise-robust representations of complex structures. Deep neural networks (DNNs) learn millions of parameters associated with a series of transformations defined by the model architecture, resulting in high-dimensional, difficult-to-interpret internal representations of input data. As DNNs become more ubiquitous across multiple sectors of our society, there is increasing recognition that mathematical methods are needed to aid analysts, researchers, and practitioners in understanding and interpreting how these models' internal representations relate to the final classification. In this paper, we apply cutting edge techniques from TDA with the goal of gaining insight into the interpretability of convolutional neural networks used for image classification. We use two common TDA approaches to explore several methods for modeling hidden-layer activations as high-dimensional point clouds, and provide experimental evidence that these point clouds capture valuable structural information about the model's process. First, we demonstrate that a distance metric based on persistent homology can be used to quantify meaningful differences between layers, and we discuss these distances in the broader context of existing representational similarity metrics for neural network interpretability. Second, we show that a mapper graph can provide semantic insight into how these models organize hierarchical class knowledge at each layer. These observations demonstrate that TDA is a useful tool to help deep learning practitioners unlock the hidden structures of their models.