# Topology of deep neural networks

arXiv: https://arxiv.org/abs/2004.06093 · HTML: https://mlsanity.com/paper/2004.06093/

## Key facts

- Cited by 112 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Apr 2020.

## Built on by (most cited first, 30 of 112)

- [Position: Topological Deep Learning is the New Frontier for Relational Learning](https://mlsanity.com/paper/2402.08871/) (2024)
- [Understanding Softmax Confidence and Uncertainty](https://mlsanity.com/paper/2106.04972/) (2021)
- [Neural Collapse with Normalized Features: A Geometric Analysis over the Riemannian Manifold](https://mlsanity.com/paper/2209.09211/) (2022)
- [Artificial Text Detection via Examining the Topology of Attention Maps](https://mlsanity.com/paper/2109.04825/) (2021)
- [Topological Detection of Trojaned Neural Networks](https://mlsanity.com/paper/2106.06469/) (2021)
- [Topological Deep Learning: A Review of an Emerging Paradigm](https://mlsanity.com/paper/2302.03836/) (2023)
- [A Review on Deep Learning in UAV Remote Sensing](https://mlsanity.com/paper/2101.10861/) (2021)
- [What are you sinking? A geometric approach on attention sink](https://mlsanity.com/paper/2508.02546/) (2025)
- [RipsNet: a general architecture for fast and robust estimation of the persistent homology of point clouds](https://mlsanity.com/paper/2202.01725/) (2022)
- [On the effectiveness of persistent homology](https://mlsanity.com/paper/2206.10551/) (2022)
- [Persformer: A Transformer Architecture for Topological Machine Learning](https://mlsanity.com/paper/2112.15210/) (2021)
- [Activation Landscapes as a Topological Summary of Neural Network Performance](https://mlsanity.com/paper/2110.10136/) (2021)
- [Learning Global Hypothesis Space for Enhancing Synergistic Reasoning Chain](https://mlsanity.com/paper/2602.09794/) (2026)
- [SPADE: A Spectral Method for Black-Box Adversarial Robustness Evaluation](https://mlsanity.com/paper/2102.03716/) (2021)
- [Topology and geometry of data manifold in deep learning](https://mlsanity.com/paper/2204.08624/) (2022)
- [Persistent Topological Features in Large Language Models](https://mlsanity.com/paper/2410.11042/) (2024)
- [Beyond Scalars: Concept-Based Alignment Analysis in Vision Transformers](https://mlsanity.com/paper/2412.06639/) (2024)
- [Topological Expressivity of ReLU Neural Networks](https://mlsanity.com/paper/2310.11130/) (2023)
- [Topological Data Analysis for Speech Processing](https://mlsanity.com/paper/2211.17223/) (2022)
- [Cycle Registration in Persistent Homology with Applications in Topological Bootstrap](https://mlsanity.com/paper/2101.00698/) (2021)
- [Characterizing and Measuring the Similarity of Neural Networks with Persistent Homology](https://mlsanity.com/paper/2101.07752/) (2021)
- [Quantitative Performance Assessment of CNN Units via Topological Entropy Calculation](https://mlsanity.com/paper/2103.09716/) (2021)
- [Reachable Polyhedral Marching (RPM): An Exact Analysis Tool for Deep-Learned Control Systems](https://mlsanity.com/paper/2210.08339/) (2022)
- [The Resurrection of the ReLU](https://mlsanity.com/paper/2505.22074/) (2025)
- [Topological Deep Learning: Classification Neural Networks](https://mlsanity.com/paper/2102.08354/) (2021)
- [Transformers with Learnable Activation Functions](https://mlsanity.com/paper/2208.14111/) (2022)
- [FONDUE: an algorithm to find the optimal dimensionality of the latent representations of variational autoencoders](https://mlsanity.com/paper/2209.12806/) (2022)
- [Safety Verification for Neural Networks Based on Set-boundary Analysis](https://mlsanity.com/paper/2210.04175/) (2022)
- [Deep neural networks architectures from the perspective of manifold learning](https://mlsanity.com/paper/2306.03406/) (2023)
- [On Characterizing the Evolution of Embedding Space of Neural Networks using Algebraic Topology](https://mlsanity.com/paper/2311.04592/) (2023)

## Abstract (arXiv)

We study how the topology of a data set M = Ma cup Mb subseteq ℝd, representing two classes a and b in a binary classification problem, changes as it passes through the layers of a well-trained neural network, i.e., with perfect accuracy on training set and near-zero generalization error (≈ 0.01%). The goal is to shed light on two mysteries in deep neural networks: (i) a nonsmooth activation function like ReLU outperforms a smooth one like hyperbolic tangent; (ii) successful neural network architectures rely on having many layers, even though a shallow network can approximate any function arbitrary well. We performed extensive experiments on the persistent homology of a wide range of point cloud data sets, both real and simulated. The results consistently demonstrate the following: (1) Neural networks operate by changing topology, transforming a topologically complicated data set into a topologically simple one as it passes through the layers. No matter how complicated the topology of M we begin with, when passed through a well-trained neural network f : ℝd → ℝp, there is a vast reduction in the Betti numbers of both components Ma and Mb; in fact they nearly always reduce to their lowest possible values: βkbigl(f(Mi)bigr) = 0 for k ≥ 1 and β0bigl(f(Mi)bigr) = 1, i =a, b. Furthermore, (2) the reduction in Betti numbers is significantly faster for ReLU activation than hyperbolic tangent activation as the former defines nonhomeomorphic maps that change topology, whereas the latter defines homeomorphic maps that preserve topology. Lastly, (3) shallow and deep networks transform data sets differently -- a shallow network operates mainly through changing geometry and changes topology only in its final layers, a deep one spreads topological changes more evenly across all layers.