# Fisher information flow in artificial neural networks

arXiv: https://arxiv.org/abs/2509.02407 · HTML: https://mlsanity.com/paper/2509.02407/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 78 references that are in the map.
- First posted on arXiv 2 Sep 2025.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [Deep Learning and the Information Bottleneck Principle](https://mlsanity.com/paper/1503.02406/) (2015)
- [Deep Convolutional Neural Network for Inverse Problems in Imaging](https://mlsanity.com/paper/1611.03679/) (2016)
- [Formal Limitations on the Measurement of Mutual Information](https://mlsanity.com/paper/1811.04251/) (2018)
- [Revisiting Natural Gradient for Deep Networks](https://mlsanity.com/paper/1301.3584/) (2013)
- [Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systems](https://mlsanity.com/paper/2104.13386/) (2021)
- [Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approach](https://mlsanity.com/paper/1806.01316/) (2018)
- [Information-Theoretic Generalization Bounds for Stochastic Gradient Descent](https://mlsanity.com/paper/2102.00931/) (2021)
- [Critical Learning Periods in Deep Neural Networks](https://mlsanity.com/paper/1711.08856/) (2017)
- [Nonlinear Information Bottleneck](https://mlsanity.com/paper/1705.02436/) (2017)
- [Information-theoretic generalization bounds for black-box learning algorithms](https://mlsanity.com/paper/2110.01584/) (2021)
- [On Information Plane Analyses of Neural Network Classifiers -- A Review](https://mlsanity.com/paper/2003.09671/) (2020)
- [Deep speckle correlation: a deep learning approach towards scalable imaging through scattering media](https://mlsanity.com/paper/1806.04139/) (2018)
- [Information Bottleneck: Exact Analysis of (Quantized) Neural Networks](https://mlsanity.com/paper/2106.12912/) (2021)
- [Determination of the edge of criticality in echo state networks through Fisher information maximization](https://mlsanity.com/paper/1603.03685/) (2016)
- [Empirical non-parametric estimation of the Fisher Information](https://mlsanity.com/paper/1408.1182/) (2014)
- [Cram'er-Rao bound-informed training of neural networks for quantitative MRI](https://mlsanity.com/paper/2109.10535/) (2021)
- [A Pessimistic Approximation for the Fisher Information Measure](https://mlsanity.com/paper/1508.03878/) (2015)
- [Maximum information states for coherent scattering measurements](https://mlsanity.com/paper/2002.10388/) (2020)
- [Sensitivity Analysis for Binary Sampling Systems via Quantitative Fisher Information Lower Bounds](https://mlsanity.com/paper/1512.03473/) (2015)

## Built on by (most cited first, 3 of 3)

- [Information Theory in Open-world Machine Learning Foundations, Frameworks, and Future Direction](https://mlsanity.com/paper/2510.15422/) (2025)
- [O(n) alternative to Quantum Fourier Transform with efficient neural net classical post-processing](https://mlsanity.com/paper/2605.16998/) (2026)
- [Speed Limit for Information Acquisition in Stochastic Learning Dynamics](https://mlsanity.com/paper/2609.08219/) (2026)

## Abstract (arXiv)

The estimation of continuous parameters from measured data plays a central role in many fields of physics. A key tool in understanding and improving such estimation processes is the concept of Fisher information, which quantifies how information about unknown parameters propagates through a physical system and determines the ultimate limits of precision. With Artificial Neural Networks (ANNs) gradually becoming an integral part of many measurement systems, it is essential to understand how they process and transmit parameter-relevant information internally. Here, we present a method to monitor the flow of Fisher information through an ANN performing a parameter estimation task, tracking it from the input to the output layer. We show that optimal estimation performance corresponds to the maximal transmission of Fisher information, and that training beyond this point results in information loss due to overfitting. This provides a model-free stopping criterion for network training-eliminating the need for a separate validation dataset. To demonstrate the practical relevance of our approach, we apply it to a network trained on data from an imaging experiment, highlighting its effectiveness in a realistic physical setting.