mlsanityData to 14 Oct 2026

Statistical Mechanics and Artificial Neural Networks: Principles, Models, and Applications

arXiv 2405.10957 ↗First posted 5 Apr 2024cond-mat.dis-nn cond-mat.stat-mech cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

The field of neuroscience and the development of artificial neural networks (ANNs) have mutually influenced each other, drawing from and contributing to many concepts initially developed in statistical mechanics. Notably, Hopfield networks and Boltzmann machines are versions of the Ising model, a model extensively studied in statistical mechanics for over a century. In the first part of this chapter, we provide an overview of the principles, models, and applications of ANNs, highlighting their connections to statistical mechanics and statistical learning theory. Artificial neural networks can be seen as high-dimensional mathematical functions, and understanding the geometric properties of their loss landscapes (i.e., the high-dimensional space on which one wishes to find extrema or saddles) can provide valuable insights into their optimization behavior, generalization abilities, and overall performance. Visualizing these functions can help us design better optimization methods and improve their generalization abilities. Thus, the second part of this chapter focuses on quantifying geometric properties and visualizing loss functions associated with deep ANNs.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 34 of its 141 references that are in the map. First posted on arXiv 5 Apr 2024.

Builds on

34 of 141 references.

References · 34

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018Neural Ordinary Differential Equationscited by 3,779
  3. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
  4. 2017Visualizing the Loss Landscape of Neural Netscited by 1,130
  5. 2014Deep Learning in Neural Networks: An Overviewcited by 770
  6. 2015Train faster, generalize better: Stability of stochastic gradient descentcited by 552
  7. 2020Hopfield Networks is All You Needcited by 475
  8. 2014New insights and perspectives on the natural gradient methodcited by 443
  9. 2019PyHessian: Neural Networks Through the Lens of the Hessiancited by 292
  10. 2014Identifying and attacking the saddle point problem in high-dimensional non-convex optimizationcited by 289
  11. 2014Qualitatively characterizing neural network optimization problemscited by 188
  12. 2015Statistical physics of inference: Thresholds and algorithmscited by 128
22 more
  1. 2018A high-bias, low-variance introduction to Machine Learning for physicistscited by 108
  2. 2019On Empirical Comparisons of Optimizers for Deep Learningcited by 103
  3. 2020Minimum Width for Universal Approximationcited by 94
  4. 2019Visualizing and Understanding the Effectiveness of BERTcited by 67
  5. 2017Data-driven discovery of Koopman eigenfunctions for controlcited by 64
  6. 2021Neural ODE control for classification, approximation and transportcited by 57
  7. 2020The gap between theory and practice in function approximation with deep neural networkscited by 54
  8. 2021Optimal control of PDEs using physics-informed neural networkscited by 52
  9. 2021Hessian Eigenspectra of More Realistic Nonlinear Modelscited by 33
  10. 2021Implicit energy regularization of neural ordinary-differential-equation controlcited by 29
  11. 2020Neural Ordinary Differential Equation Control of Dynamics on Graphscited by 24
  12. 2022Interpretable Polynomial Neural Ordinary Differential Equationscited by 24
  13. 2022Visualizing high-dimensional loss landscapes with Hessian directionscited by 16
  14. 2021Exploring the Geometry and Topology of Neural Network Loss Landscapescited by 14
  15. 2020Learning the Ising Model with Generative Neural Networkscited by 9
  16. 2021Supervised perceptron learning vs unsupervised Hebbian unlearning: Approaching optimal memory retrieval in Hopfield-like networkscited by 9
  17. 2022Spectrally Adapted Physics-Informed Neural Networks for Solving Unbounded Domain Problemscited by 9
  18. 2022Near-optimal control of dynamical systems with neural ordinary differential equationscited by 9
  19. 2022Complex networks with complex weightscited by 9
  20. 2023Gradient-free training of neural ODEs for system identification and control using ensemble Kalman inversioncited by 9
  21. 2022Control of Dual-Sourcing Inventory Systems using Recurrent Neural Networkscited by 6
  22. 2024Control of Medical Digital Twins with Artificial Neural Networkscited by 3

Built on by

Not cited yet.