# Estimating Information Flow in Deep Neural Networks

arXiv: https://arxiv.org/abs/1810.05728 · HTML: https://mlsanity.com/paper/1810.05728/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Oct 2018.

## Built on by (most cited first, 7 of 7)

- [To Compress or Not to Compress- Self-Supervised Learning and Information Theory: A Review](https://mlsanity.com/paper/2304.09355/) (2023)
- [An Information-Theoretic Perspective on Variance-Invariance-Covariance Regularization](https://mlsanity.com/paper/2303.00633/) (2023)
- [An Information Theory-inspired Strategy for Automatic Network Pruning](https://mlsanity.com/paper/2108.08532/) (2021)
- [What Do We Maximize in Self-Supervised Learning?](https://mlsanity.com/paper/2207.10081/) (2022)
- [Gacs-Korner Common Information Variational Autoencoder](https://mlsanity.com/paper/2205.12239/) (2022)
- [A representation-learning game for classes of prediction tasks](https://mlsanity.com/paper/2403.06971/) (2024)
- [k-Sliced Mutual Information: A Quantitative Study of Scalability with Dimension](https://mlsanity.com/paper/2206.08526/) (2022)

## Abstract (arXiv)

We study the flow of information and the evolution of internal representations during deep neural network (DNN) training, aiming to demystify the compression aspect of the information bottleneck theory. The theory suggests that DNN training comprises a rapid fitting phase followed by a slower compression phase, in which the mutual information I(X;T) between the input X and internal representations T decreases. Several papers observe compression of estimated mutual information on different DNN models, but the true I(X;T) over these networks is provably either constant (discrete X) or infinite (continuous X). This work explains the discrepancy between theory and experiments, and clarifies what was actually measured by these past works. To this end, we introduce an auxiliary (noisy) DNN framework for which I(X;T) is a meaningful quantity that depends on the network's parameters. This noisy framework is shown to be a good proxy for the original (deterministic) DNN both in terms of performance and the learned representations. We then develop a rigorous estimator for I(X;T) in noisy DNNs and observe compression in various models. By relating I(X;T) in the noisy DNN to an information-theoretic communication problem, we show that compression is driven by the progressive clustering of hidden representations of inputs from the same class. Several methods to directly monitor clustering of hidden representations, both in noisy and deterministic DNNs, are used to show that meaningful clusters form in the T space. Finally, we return to the estimator of I(X;T) employed in past works, and demonstrate that while it fails to capture the true (vacuous) mutual information, it does serve as a measure for clustering. This clarifies the past observations of compression and isolates the geometric clustering of hidden representations as the true phenomenon of interest.