# Which bits went where? Past and future transfer entropy decomposition with the information bottleneck

arXiv: https://arxiv.org/abs/2411.04992 · HTML: https://mlsanity.com/paper/2411.04992/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 22 references that are in the map.
- First posted on arXiv 7 Nov 2024.

## Builds on (most cited first, 8 of 8)

- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [The Information Bottleneck Problem and Its Applications in Machine Learning](https://mlsanity.com/paper/2004.14941/) (2020)
- [CCMI : Classifier based Conditional Mutual Information Estimation](https://mlsanity.com/paper/1906.01824/) (2019)
- [Information decomposition in complex systems via machine learning](https://mlsanity.com/paper/2307.04755/) (2023)
- [Interpretability with full complexity by constraining feature information](https://mlsanity.com/paper/2211.17264/) (2022)
- [Transfer Entropy Bottleneck: Learning Sequence to Sequence Information Transfer](https://mlsanity.com/paper/2211.16607/) (2022)
- [Inferring network properties from time series using transfer entropy and mutual information: validation of multivariate versus bivariate approaches](https://mlsanity.com/paper/2007.07500/) (2020)

## Built on by (most cited first, 2 of 2)

- [Rethinking Spatio-Temporal Anomaly Detection: A Vision for Causality-Driven Cybersecurity](https://mlsanity.com/paper/2507.08177/) (2025)
- [InfoDPCCA: Information-Theoretic Dynamic Probabilistic Canonical Correlation Analysis](https://mlsanity.com/paper/2506.08884/) (2025)

## Abstract (arXiv)

Whether the system under study is a shoal of fish, a collection of neurons, or a set of interacting atmospheric and oceanic processes, transfer entropy measures the flow of information between time series and can detect possible causal relationships. Much like mutual information, transfer entropy is generally reported as a single value summarizing an amount of shared variation, yet a more fine-grained accounting might illuminate much about the processes under study. Here we propose to decompose transfer entropy and localize the bits of variation on both sides of information flow: that of the originating process's past and that of the receiving process's future. We employ the information bottleneck (IB) to compress the time series and identify the transferred entropy. We apply our method to decompose the transfer entropy in several synthetic recurrent processes and an experimental mouse dataset of concurrent behavioral and neural activity. Our approach highlights the nuanced dynamics within information flow, laying a foundation for future explorations into the intricate interplay of temporal processes in complex systems.