mlsanityData to 7 Oct 2026

Decomposing neural networks as mappings of correlation functions

arXiv 2202.04925 ↗First posted 10 Feb 2022cond-mat.dis-nn stat.MLcited by 12

Not on the map. Why?

Abstract (arXiv)

Understanding the functional principles of information processing in deep neural networks continues to be a challenge, in particular for networks with trained and thus non-random weights. To address this issue, we study the mapping between probability distributions implemented by a deep feed-forward network. We characterize this mapping as an iterated transformation of distributions, where the non-linearity in each layer transfers information between different orders of correlation functions. This allows us to identify essential statistics in the data, as well as different information representations that can be used by neural networks. Applied to an XOR task and to MNIST, we show that correlations up to second order predominantly capture the information processing in the internal layers, while the input layer also extracts higher-order correlations from the data. This analysis provides a quantitative and explainable perspective on classification.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 23 of its 69 references that are in the map. First posted on arXiv 10 Feb 2022.

Builds on

23 of 69 references.

References · 23

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2016Wide Residual Networkscited by 3,057
  4. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  5. 2017Deep Neural Networks as Gaussian Processescited by 617
  6. 2017Opening the Black Box of Deep Neural Networks via Informationcited by 494
  7. 2016On the Expressive Power of Deep Neural Networkscited by 302
  8. 2016Exponential expressivity in deep neural networks through transient chaoscited by 260
  9. 2016Deep Information Propagationcited by 194
  10. 2018A mathematical theory of semantic development in deep neural networkscited by 182
  11. 2021The Principles of Deep Learning Theorycited by 169
  12. 2018Deep Convolutional Networks as shallow Gaussian Processescited by 137
11 more
  1. 2021Learning curves of generic features maps for realistic datasets with a teacher-student modelcited by 126
  2. 2020The Gaussian equivalence of generative models for learning with shallow neural networkscited by 113
  3. 2016Why does deep and cheap learning work so well?cited by 100
  4. 2019Non-Gaussian processes and neural networks at finite widthscited by 78
  5. 2019Asymptotics of Wide Networks from Feynman Diagramscited by 77
  6. 2020Random Matrix Theory Proves that Deep Learning Representations of GAN-data Behave as Gaussian Mixturescited by 47
  7. 2020Modeling from Features: a Mean-field Framework for Over-parameterized Deep Neural Networkscited by 44
  8. 2020Predicting the outputs of finite deep neural networks trained with noisy gradientscited by 31
  9. 2021Unified field theoretical approach to deep and recurrent neuronal networkscited by 20
  10. 2016Functional methods for disordered neural networkscited by 4
  11. 2020Weakly-correlated synapses promote dimension reduction in deep neural networkscited by 4

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 3; 2024: 2; 2025: 5.

12 papers.

  1. 2022Neural networks trained with SGD learn distributions of increasing complexitycited by 50
  2. 2023Bayes-optimal Learning of Deep Random Networks of Extensive-widthcited by 41
  3. 2023Neural Network Field Theories: Non-Gaussianity, Actions, and Localitycited by 21
  4. 2023Bayesian Renormalizationcited by 13
  5. 2024Sliding down the stairs: how correlated latent variables accelerate learning with neural networkscited by 12
  6. 2025Interpretable Machine Learning in Physics: A Reviewcited by 11
  7. 2022Origami in N dimensions: How feed-forward networks manufacture linear separabilitycited by 9
  8. 2025Generalization Dynamics of Linear Diffusion Modelscited by 9
  9. 2024NCoder -- A Quantum Field Theory approach to encoding datacited by 7
  10. 2025Flat Channels to Infinity in Neural Loss Landscapescited by 3
  11. 2025Probing Geometry of Next Token Prediction Using Cumulant Expansion of the Softmax Entropycited by 0
  12. 2025Symmetry and Generalisation in Neural Approximations of Renormalisation Transformationscited by 0