mlsanityData to 14 Oct 2026

A Spin Glass Characterization of Neural Networks

arXiv 2508.07397 ↗First posted 10 Aug 2025cond-mat.dis-nn cs.AI cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

This work presents a statistical mechanics characterization of neural networks, motivated by the replica symmetry breaking (RSB) phenomenon in spin glasses. A Hopfield-type spin glass model is constructed from a given feedforward neural network (FNN). Overlaps between simulated replica samples serve as a characteristic descriptor of the FNN. The connection between the spin-glass description and commonly studied properties of the FNN -- such as data fitting, capacity, generalization, and robustness -- has been investigated and empirically demonstrated. Unlike prior analytical studies that focus on model ensembles, this method provides a computable descriptor for individual network instances, which reveals nontrivial structural properties that are not captured by conventional metrics such as loss or accuracy. Preliminary results suggests its potential for practical applications such as model inspection, safety verification, and detection of hidden vulnerabilities.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 26 of its 53 references that are in the map. First posted on arXiv 10 Aug 2025.

Builds on

26 of 53 references.

References · 26

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2023Segment Anythingcited by 8,234
  6. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  7. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,150
  8. 2017Deep Neural Networks as Gaussian Processescited by 622
  9. 2016On the Expressive Power of Deep Neural Networkscited by 306
  10. 2017Stochastic Gradient Descent as Approximate Bayesian Inferencecited by 300
  11. 2014Identifying and attacking the saddle point problem in high-dimensional non-convex optimizationcited by 289
  12. 2014The Loss Surfaces of Multilayer Networkscited by 280
14 more
  1. 2016Exponential expressivity in deep neural networks through transient chaoscited by 261
  2. 2016Deep Information Propagationcited by 195
  3. 2018Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networkscited by 176
  4. 2015Statistical physics of inference: Thresholds and algorithmscited by 128
  5. 2018Stochastic Modified Equations and Dynamics of Stochastic Gradient Algorithms I: Mathematical Foundationscited by 114
  6. 2020Towards a Mathematical Understanding of Neural Network-Based Machine Learning: what we know and what we don'tcited by 95
  7. 2018Entropy and mutual information in models of deep neural networkscited by 51
  8. 2023Unifying Grokking and Double Descentcited by 50
  9. 2020On Learning Rates and Schr"odinger Operatorscited by 33
  10. 2022Stochastic gradient descent introduces an effective landscape-dependent regularization favoring flat solutionscited by 22
  11. 2021Landscape analysis for shallow neural networks: complete classification of critical points for affine target functionscited by 11
  12. 2021Edge of chaos as a guiding principle for modern neural network trainingcited by 10
  13. 2023Quantum Langevin Dynamics for Optimizationcited by 7
  14. 2023Bridging Lottery Ticket and Grokking: Understanding Grokking from Inner Structure of Networkscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Visualizing the loss landscapes of physics-informed neural networkscited by 1