# Exploring higher-order neural network node interactions with total correlation

arXiv: https://arxiv.org/abs/2402.04440 · HTML: https://mlsanity.com/paper/2402.04440/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 31 references that are in the map.
- First posted on arXiv 6 Feb 2024.

## Builds on (most cited first, 10 of 10)

- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Knowledge Neurons in Pretrained Transformers](https://mlsanity.com/paper/2104.08696/) (2021)
- [Natural Language Descriptions of Deep Visual Features](https://mlsanity.com/paper/2201.11114/) (2022)
- [Editing a classifier by rewriting its prediction rules](https://mlsanity.com/paper/2112.01008/) (2021)
- [Robust Feature-Level Adversaries are Interpretability Tools](https://mlsanity.com/paper/2110.03605/) (2021)
- [Higher-order interactions shape collective dynamics differently in hypergraphs and simplicial complexes](https://mlsanity.com/paper/2203.03060/) (2022)
- [Diagnostics for Deep Neural Networks with Automated Copy/Paste Attacks](https://mlsanity.com/paper/2211.10024/) (2022)
- [Discovering Structure in High-Dimensional Data Through Correlation Explanation](https://mlsanity.com/paper/1406.1222/) (2014)
- [Maximally Informative Hierarchical Representations of High-Dimensional Data](https://mlsanity.com/paper/1410.7404/) (2014)
- [Explosive cooperation in social dilemmas on higher-order networks](https://mlsanity.com/paper/2303.11475/) (2023)

## Abstract (arXiv)

In domains such as ecological systems, collaborations, and the human brain the variables interact in complex ways. Yet accurately characterizing higher-order variable interactions (HOIs) is a difficult problem that is further exacerbated when the HOIs change across the data. To solve this problem we propose a new method called Local Correlation Explanation (CorEx) to capture HOIs at a local scale by first clustering data points based on their proximity on the data manifold. We then use a multivariate version of the mutual information called the total correlation, to construct a latent factor representation of the data within each cluster to learn the local HOIs. We use Local CorEx to explore HOIs in synthetic and real world data to extract hidden insights about the data structure. Lastly, we demonstrate Local CorEx's suitability to explore and interpret the inner workings of trained neural networks.