# Information-Theoretic Measures on Lattices for High-Order Interactions

arXiv: https://arxiv.org/abs/2408.07533 · HTML: https://mlsanity.com/paper/2408.07533/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 83 references that are in the map.
- First posted on arXiv 14 Aug 2024.

## Builds on (most cited first, 20 of 20)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [MINE: Mutual Information Neural Estimation](https://mlsanity.com/paper/1801.04062/) (2018)
- [Networks beyond pairwise interactions: structure and dynamics](https://mlsanity.com/paper/2006.01764/) (2020)
- [Higher-order organization of complex networks](https://mlsanity.com/paper/1612.08447/) (2016)
- [What are higher-order networks?](https://mlsanity.com/paper/2104.11329/) (2021)
- [Nonparametric independence testing via mutual information](https://mlsanity.com/paper/1711.06642/) (2017)
- [Hypergraph reconstruction from network data](https://mlsanity.com/paper/2008.04948/) (2020)
- [Unveiling the higher-order organization of multivariate time series](https://mlsanity.com/paper/2203.10702/) (2022)
- [Information Theoretical Estimators Toolbox](https://mlsanity.com/paper/1405.2106/) (2014)
- [Two-temperature logistic regression based on the Tsallis divergence](https://mlsanity.com/paper/1705.07210/) (2017)
- [Interpretable statistical representations of neural population dynamics and geometry](https://mlsanity.com/paper/2304.03376/) (2023)
- [Neural Feature Learning in Function Space](https://mlsanity.com/paper/2309.10140/) (2023)
- [Variational Inference MPC using Tsallis Divergence](https://mlsanity.com/paper/2104.00241/) (2021)
- [Multivariate Information Theory Uncovers Synergistic Subsystems of the Human Cerebral Cortex](https://mlsanity.com/paper/2206.06477/) (2022)
- [Partial entropy decomposition reveals higher-order structures in human brain activity](https://mlsanity.com/paper/2301.05307/) (2023)
- [The Partial Entropy Decomposition: Decomposing multivariate entropy and mutual information via pointwise common surprisal](https://mlsanity.com/paper/1702.01591/) (2017)
- [Kernelized Cumulants: Beyond Kernel Mean Embeddings](https://mlsanity.com/paper/2301.12466/) (2023)
- [Multivariate Dependence Beyond Shannon Information](https://mlsanity.com/paper/1609.01233/) (2016)
- [Kernel-based Joint Independence Tests for Multivariate Stationary and Non-stationary Time Series](https://mlsanity.com/paper/2305.08529/) (2023)
- [Interaction Measures, Partition Lattices and Kernel Tests for High-Order Interactions](https://mlsanity.com/paper/2306.00904/) (2023)

## Built on by (most cited first, 3 of 3)

- [Permutation-Free High-Order Interaction Tests](https://mlsanity.com/paper/2506.05963/) (2025)
- [Information-theoretic signatures of causality in Bayesian networks and hypergraphs](https://mlsanity.com/paper/2512.20552/) (2025)
- [DIPHINE: Diffusion-based Phi-ID Neural Estimator](https://mlsanity.com/paper/2606.18997/) (2026)

## Abstract (arXiv)

Traditional measures based solely on pairwise associations often fail to capture the complex statistical structure of multivariate data. Existing approaches for identifying information shared among d>3 variables are frequently computationally intractable, asymmetric with respect to a target variable, or unable to account for all the ways in which the joint probability distribution can be factorised. Here we present a systematic framework based on lattice theory to derive higher-order information-theoretic measures for multivariate data. Our construction uses lattice and operator function pairs, whereby an operator function is applied over a lattice that represents the algebraic relationships among variables. We show that many commonly used measures can be derived within this framework, yet they fail to capture all interactions for d>3, either because they are defined on restricted sublattices, or because the use of the KL divergence as an operator function, a typical choice, leads to undesired disregard of groups of interactions. To fully characterise all interactions among d variables, we introduce the Streitberg Information, which is defined over the full partition lattice and uses generalised divergences (beyond KL) as operator functions. We validate the Streitberg Information on synthetic data, and illustrate its application in detecting complex interactions among stocks, decoding neural signals, and performing feature selection in machine learning.