# ML-PWS: Estimating the Mutual Information Between Experimental Time Series Using Neural Networks

arXiv: https://arxiv.org/abs/2508.16509 · HTML: https://mlsanity.com/paper/2508.16509/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 55 references that are in the map.
- First posted on arXiv 22 Aug 2025.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Pixel Recurrent Neural Networks](https://mlsanity.com/paper/1601.06759/) (2016)
- [Masked Autoregressive Flow for Density Estimation](https://mlsanity.com/paper/1705.07057/) (2017)
- [On Variational Bounds of Mutual Information](https://mlsanity.com/paper/1905.06922/) (2019)
- [Generating Sequences With Recurrent Neural Networks](https://mlsanity.com/paper/1308.0850/) (2013)
- [Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design](https://mlsanity.com/paper/1902.00275/) (2019)
- [Formal Limitations on the Measurement of Mutual Information](https://mlsanity.com/paper/1811.04251/) (2018)
- [Efficient Estimation of Mutual Information for Strongly Dependent Variables](https://mlsanity.com/paper/1411.2003/) (2014)
- [High accuracy decoding of dynamical motion from a large retinal population](https://mlsanity.com/paper/1408.3028/) (2014)

## Abstract (arXiv)

The ability to quantify information transmission is crucial for the analysis and design of both natural and engineered systems. For systems driven by time-varying signals, the fundamental measure is the information transmission rate. However, due to the high dimensionality of signal trajectory space, this rate cannot be obtained directly from time-series data without approximations. Path Weight Sampling (PWS) is a computational technique that enables the exact calculation of the information rate for any stochastic model, raising the question of how this rate can be determined from time-series data in the absence of a prior model. Here, we present a method that combines machine learning (ML) with PWS: a generative model is learned from time-series data, to which PWS is applied to yield a rigorous lower bound on the information rate. We demonstrate the accuracy of this technique, called ML-PWS, by comparing its results on synthetic time-series data generated from several non-linear models against ground-truth results obtained by applying PWS directly to the same models. We illustrate the utility of ML-PWS by applying it to neuronal time-series data.