# Decorrelating neurons using persistence

arXiv: https://arxiv.org/abs/2308.04870 · HTML: https://mlsanity.com/paper/2308.04870/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 51 references that are in the map.
- First posted on arXiv 9 Aug 2023.

## Builds on (most cited first, 27 of 27)

- [Fantastic Generalization Measures and Where to Find Them](https://mlsanity.com/paper/1912.02178/) (2019)
- [Topology-Preserving Deep Image Segmentation](https://mlsanity.com/paper/1906.05404/) (2019)
- [Generalization in Deep Learning](https://mlsanity.com/paper/1710.05468/) (2017)
- [A Topological Loss Function for Deep-Learning based Image Segmentation using Persistent Homology](https://mlsanity.com/paper/1910.01877/) (2019)
- [PersLay: A Neural Network Layer for Persistence Diagrams and New Graph Topological Signatures](https://mlsanity.com/paper/1904.09378/) (2019)
- [Deep Learning with Topological Signatures](https://mlsanity.com/paper/1707.04041/) (2017)
- [Reducing Overfitting in Deep Networks by Decorrelating Representations](https://mlsanity.com/paper/1511.06068/) (2015)
- [A Topology Layer for Machine Learning](https://mlsanity.com/paper/1905.12200/) (2019)
- [In Search of Robust Measures of Generalization](https://mlsanity.com/paper/2010.11924/) (2020)
- [Topology-Aware Segmentation Using Discrete Morse Theory](https://mlsanity.com/paper/2103.09992/) (2021)
- [Neural Persistence: A Complexity Measure for Deep Neural Networks Using Algebraic Topology](https://mlsanity.com/paper/1812.09764/) (2018)
- [Intrinsic Dimension, Persistent Homology and Generalization in Neural Networks](https://mlsanity.com/paper/2111.13171/) (2021)
- [A Topological Regularizer for Classifiers via Persistent Homology](https://mlsanity.com/paper/1806.10714/) (2018)
- [Computing the Testing Error without a Testing Set](https://mlsanity.com/paper/2005.00450/) (2020)
- [Connectivity-Optimized Representation Learning via Persistent Homology](https://mlsanity.com/paper/1906.09003/) (2019)
- [NeurIPS 2020 Competition: Predicting Generalization in Deep Learning](https://mlsanity.com/paper/2012.07976/) (2020)
- [How does Weight Correlation Affect the Generalisation Ability of Deep Neural Networks](https://mlsanity.com/paper/2010.05983/) (2020)
- [Topological Approaches to Deep Learning](https://mlsanity.com/paper/1811.01122/) (2018)
- [A Framework for Differential Calculus on Persistence Barcodes](https://mlsanity.com/paper/1910.00960/) (2019)
- [GPU-Accelerated Computation of Vietoris-Rips Persistence Barcodes](https://mlsanity.com/paper/2003.07989/) (2020)
- [Persformer: A Transformer Architecture for Topological Machine Learning](https://mlsanity.com/paper/2112.15210/) (2021)
- [A persistent homology-based topological loss function for multi-class CNN segmentation of cardiac MRI](https://mlsanity.com/paper/2008.09585/) (2020)
- [giotto-ph: A Python Library for High-Performance Computation of Persistent Homology of Vietoris-Rips Filtrations](https://mlsanity.com/paper/2107.05412/) (2021)
- [On Measuring Excess Capacity in Neural Networks](https://mlsanity.com/paper/2202.08070/) (2022)
- [Topologically Densified Distributions](https://mlsanity.com/paper/2002.04805/) (2020)
- [Neural circuit function redundancy in brain disorders](https://mlsanity.com/paper/2102.03039/) (2021)
- [Engineering Massively Parallel MST Algorithms](https://mlsanity.com/paper/2302.12199/) (2023)

## Abstract (arXiv)

We propose a novel way to improve the generalisation capacity of deep learning models by reducing high correlations between neurons. For this, we present two regularisation terms computed from the weights of a minimum spanning tree of the clique whose vertices are the neurons of a given network (or a sample of those), where weights on edges are correlation dissimilarities. We provide an extensive set of experiments to validate the effectiveness of our terms, showing that they outperform popular ones. Also, we demonstrate that naive minimisation of all correlations between neurons obtains lower accuracies than our regularisation terms, suggesting that redundancies play a significant role in artificial neural networks, as evidenced by some studies in neuroscience for real networks. We include a proof of differentiability of our regularisers, thus developing the first effective topological persistence-based regularisation terms that consider the whole set of neurons and that can be applied to a feedforward architecture in any deep learning task such as classification, data generation, or regression.