mlsanityData to 7 Oct 2026

Fisher information flow in artificial neural networks

arXiv 2509.02407 ↗First posted 2 Sep 2025cs.LG physics.data-ancited by 3

Not on the map. Why?

Abstract (arXiv)

The estimation of continuous parameters from measured data plays a central role in many fields of physics. A key tool in understanding and improving such estimation processes is the concept of Fisher information, which quantifies how information about unknown parameters propagates through a physical system and determines the ultimate limits of precision. With Artificial Neural Networks (ANNs) gradually becoming an integral part of many measurement systems, it is essential to understand how they process and transmit parameter-relevant information internally. Here, we present a method to monitor the flow of Fisher information through an ANN performing a parameter estimation task, tracking it from the input to the output layer. We show that optimal estimation performance corresponds to the maximal transmission of Fisher information, and that training beyond this point results in information loss due to overfitting. This provides a model-free stopping criterion for network training-eliminating the need for a separate validation dataset. To demonstrate the practical relevance of our approach, we apply it to a network trained on data from an imaging experiment, highlighting its effectiveness in a realistic physical setting.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 22 of its 78 references that are in the map. First posted on arXiv 2 Sep 2025.

Builds on

22 of 78 references.

References · 22

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  3. 2016Deep Variational Information Bottleneckcited by 915
  4. 2015Deep Learning and the Information Bottleneck Principlecited by 793
  5. 2016Deep Convolutional Neural Network for Inverse Problems in Imagingcited by 431
  6. 2018Formal Limitations on the Measurement of Mutual Informationcited by 189
  7. 2013Revisiting Natural Gradient for Deep Networkscited by 157
  8. 2021Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systemscited by 125
  9. 2018Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approachcited by 89
  10. 2021Information-Theoretic Generalization Bounds for Stochastic Gradient Descentcited by 84
  11. 2017Critical Learning Periods in Deep Neural Networkscited by 78
  12. 2017Nonlinear Information Bottleneckcited by 71
10 more
  1. 2021Information-theoretic generalization bounds for black-box learning algorithmscited by 49
  2. 2020On Information Plane Analyses of Neural Network Classifiers -- A Reviewcited by 35
  3. 2018Deep speckle correlation: a deep learning approach towards scalable imaging through scattering mediacited by 23
  4. 2021Information Bottleneck: Exact Analysis of (Quantized) Neural Networkscited by 14
  5. 2016Determination of the edge of criticality in echo state networks through Fisher information maximizationcited by 12
  6. 2014Empirical non-parametric estimation of the Fisher Informationcited by 5
  7. 2021Cram'er-Rao bound-informed training of neural networks for quantitative MRIcited by 3
  8. 2015A Pessimistic Approximation for the Fisher Information Measurecited by 2
  9. 2020Maximum information states for coherent scattering measurementscited by 2
  10. 2015Sensitivity Analysis for Binary Sampling Systems via Quantitative Fisher Information Lower Boundscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Information Theory in Open-world Machine Learning Foundations, Frameworks, and Future Directioncited by 0
  2. 2026O(n) alternative to Quantum Fourier Transform with efficient neural net classical post-processingcited by 0
  3. 2026Speed Limit for Information Acquisition in Stochastic Learning Dynamicscited by 0