Information flow in multilayer perceptrons: an in-depth analysis
Not on the map. Why?
Abstract (arXiv)
Analysing how information flows along the layers of a multilayer perceptron is a topic of paramount importance in the field of artificial neural networks. After framing the problem from the point of view of information theory, in this position article a specific investigation is conducted on the way information is processed, with particular reference to the requirements imposed by supervised learning. To this end, the concept of information matrix is devised and then used as formal framework for understanding the aetiology of optimisation strategies and for studying the information flow. The underlying research for this article has also produced several key outcomes: i) the definition of a parametric optimisation strategy, ii) the finding that the optimisation strategy proposed in the information bottleneck framework shares strong similarities with the one derived from the information matrix, and iii) the insight that a multilayer perceptron serves as a kind of "adaptor", meant to process the input according to the given objective.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 8 of its 32 references that are in the map. First posted on arXiv 11 Oct 2025.
Builds on
8 of 32 references.
References · 8
- 2015Deep Learning and the Information Bottleneck Principlecited by 793
- 2017Opening the Black Box of Deep Neural Networks via Informationcited by 494
- 2018SpectralNet: Spectral Clustering using Deep Neural Networkscited by 94
- 2016The deterministic information bottleneckcited by 42
- 2019Learnability for the Information Bottleneckcited by 23
- 2017Forward Thinking: Building and Training Neural Networks One Layer at a Timecited by 15
- 2021A Critical Review of Information Bottleneck Theory and its Applications to Deep Learningcited by 3
- 2019Compression and Localization in Reinforcement Learning for ATARI Gamescited by 1
Built on by
Not cited yet.