# Information flow in multilayer perceptrons: an in-depth analysis

arXiv: https://arxiv.org/abs/2510.13846 · HTML: https://mlsanity.com/paper/2510.13846/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 32 references that are in the map.
- First posted on arXiv 11 Oct 2025.

## Builds on (most cited first, 8 of 8)

- [Deep Learning and the Information Bottleneck Principle](https://mlsanity.com/paper/1503.02406/) (2015)
- [Opening the Black Box of Deep Neural Networks via Information](https://mlsanity.com/paper/1703.00810/) (2017)
- [SpectralNet: Spectral Clustering using Deep Neural Networks](https://mlsanity.com/paper/1801.01587/) (2018)
- [The deterministic information bottleneck](https://mlsanity.com/paper/1604.00268/) (2016)
- [Learnability for the Information Bottleneck](https://mlsanity.com/paper/1907.07331/) (2019)
- [Forward Thinking: Building and Training Neural Networks One Layer at a Time](https://mlsanity.com/paper/1706.02480/) (2017)
- [A Critical Review of Information Bottleneck Theory and its Applications to Deep Learning](https://mlsanity.com/paper/2105.04405/) (2021)
- [Compression and Localization in Reinforcement Learning for ATARI Games](https://mlsanity.com/paper/1904.09489/) (2019)

## Abstract (arXiv)

Analysing how information flows along the layers of a multilayer perceptron is a topic of paramount importance in the field of artificial neural networks. After framing the problem from the point of view of information theory, in this position article a specific investigation is conducted on the way information is processed, with particular reference to the requirements imposed by supervised learning. To this end, the concept of information matrix is devised and then used as formal framework for understanding the aetiology of optimisation strategies and for studying the information flow. The underlying research for this article has also produced several key outcomes: i) the definition of a parametric optimisation strategy, ii) the finding that the optimisation strategy proposed in the information bottleneck framework shares strong similarities with the one derived from the information matrix, and iii) the insight that a multilayer perceptron serves as a kind of "adaptor", meant to process the input according to the given objective.