# Topology of Learning in Artificial Neural Networks

arXiv: https://arxiv.org/abs/1902.08160 · HTML: https://mlsanity.com/paper/1902.08160/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Feb 2019.

## Built on by (most cited first, 3 of 3)

- [Position: Topological Deep Learning is the New Frontier for Relational Learning](https://mlsanity.com/paper/2402.08871/) (2024)
- [Leveraging Persistence Image to Enhance Robustness and Performance in Curvilinear Structure Segmentation](https://mlsanity.com/paper/2601.18045/) (2026)
- [Leveraging The Topological Consistencies of Learning in Deep Neural Networks](https://mlsanity.com/paper/2111.15651/) (2021)

## Abstract (arXiv)

Understanding how neural networks learn remains one of the central challenges in machine learning research. From random at the start of training, the weights of a neural network evolve in such a way as to be able to perform a variety of tasks, like classifying images. Here we study the emergence of structure in the weights by applying methods from topological data analysis. We train simple feedforward neural networks on the MNIST dataset and monitor the evolution of the weights. When initialized to zero, the weights follow trajectories that branch off recurrently, thus generating trees that describe the growth of the effective capacity of each layer. When initialized to tiny random values, the weights evolve smoothly along two-dimensional surfaces. We show that natural coordinates on these learning surfaces correspond to important factors of variation.