# Leveraging The Topological Consistencies of Learning in Deep Neural Networks

arXiv: https://arxiv.org/abs/2111.15651 · HTML: https://mlsanity.com/paper/2111.15651/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 35 references that are in the map.
- First posted on arXiv 30 Nov 2021.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [Reconciling modern machine learning practice and the bias-variance trade-off](https://mlsanity.com/paper/1812.11118/) (2018)
- [Deep Learning for Classical Japanese Literature](https://mlsanity.com/paper/1812.01718/) (2018)
- [Topology-Preserving Deep Image Segmentation](https://mlsanity.com/paper/1906.05404/) (2019)
- [A Topological Loss Function for Deep-Learning based Image Segmentation using Persistent Homology](https://mlsanity.com/paper/1910.01877/) (2019)
- [Topological Autoencoders](https://mlsanity.com/paper/1906.00722/) (2019)
- [PersLay: A Neural Network Layer for Persistence Diagrams and New Graph Topological Signatures](https://mlsanity.com/paper/1904.09378/) (2019)
- [A Topology Layer for Machine Learning](https://mlsanity.com/paper/1905.12200/) (2019)
- [Neural Persistence: A Complexity Measure for Deep Neural Networks Using Algebraic Topology](https://mlsanity.com/paper/1812.09764/) (2018)
- [A Topological Regularizer for Classifiers via Persistent Homology](https://mlsanity.com/paper/1806.10714/) (2018)
- [The Unreasonable Effectiveness of Deep Learning in Artificial Intelligence](https://mlsanity.com/paper/2002.04806/) (2020)
- [Computing the Testing Error without a Testing Set](https://mlsanity.com/paper/2005.00450/) (2020)
- [On Characterizing the Capacity of Neural Networks using Algebraic Topology](https://mlsanity.com/paper/1802.04443/) (2018)
- [Topological Approaches to Deep Learning](https://mlsanity.com/paper/1811.01122/) (2018)
- [Towards a topological-geometrical theory of group equivariant non-expansive operators for data analysis and machine learning](https://mlsanity.com/paper/1812.11832/) (2018)
- [Exposition and Interpretation of the Topology of Neural Networks](https://mlsanity.com/paper/1810.03234/) (2018)
- [Adversary Detection in Neural Networks via Persistent Homology](https://mlsanity.com/paper/1711.10056/) (2017)
- [Characterizing and Measuring the Similarity of Neural Networks with Persistent Homology](https://mlsanity.com/paper/2101.07752/) (2021)
- [Topology of Learning in Artificial Neural Networks](https://mlsanity.com/paper/1902.08160/) (2019)

## Abstract (arXiv)

Recently, methods have been developed to accurately predict the testing performance of a Deep Neural Network (DNN) on a particular task, given statistics of its underlying topological structure. However, further leveraging this newly found insight for practical applications is intractable due to the high computational cost in terms of time and memory. In this work, we define a new class of topological features that accurately characterize the progress of learning while being quick to compute during running time. Additionally, our proposed topological features are readily equipped for backpropagation, meaning that they can be incorporated in end-to-end training. Our newly developed practical topological characterization of DNNs allows for an additional set of applications. We first show we can predict the performance of a DNN without a testing set and without the need for high-performance computing. We also demonstrate our topological characterization of DNNs is effective in estimating task similarity. Lastly, we show we can induce learning in DNNs by actively constraining the DNN's topological structure. This opens up new avenues in constricting the underlying structure of DNNs in a meta-learning framework.