mlsanityData to 7 Oct 2026

TASI Lectures on Physics for Machine Learning

arXiv 2408.00082 ↗First posted 31 Jul 2024hep-th cs.LG hep-phcited by 4

Not on the map. Why?

Abstract (arXiv)

These notes are based on lectures I gave at TASI 2024 on Physics for Machine Learning. The focus is on neural network theory, organized according to network expressivity, statistics, and dynamics. I present classic results such as the universal approximation theorem and neural network / Gaussian process correspondence, and also more recent results such as the neural tangent kernel, feature learning with the maximal update parameterization, and Kolmogorov-Arnold networks. The exposition on neural network theory emphasizes a field theoretic perspective familiar to theoretical physicists. I elaborate on connections between the two, including a neural network approach to field theory.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 31 of its 58 references that are in the map. First posted on arXiv 31 Jul 2024.

Builds on

31 of 58 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2020Scaling Laws for Neural Language Modelscited by 6,411
  7. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  8. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  9. 2016Group Equivariant Convolutional Networkscited by 1,154
  10. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
  11. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 732
  12. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 723
19 more
  1. 2021E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentialscited by 701
  2. 2024KAN: Kolmogorov-Arnold Networkscited by 634
  3. 2016Deep Information Propagationcited by 194
  4. 2020Feature Learning in Infinite-Width Neural Networkscited by 182
  5. 2019Tensor Programs I: Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processescited by 148
  6. 2022Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networkscited by 104
  7. 2020Sampling using SU(N) gauge equivariant flowscited by 83
  8. 2019Non-Gaussian processes and neural networks at finite widthscited by 78
  9. 2020Neural Networks and Quantum Field Theorycited by 58
  10. 20183D G-CNNs for Pulmonary Nodule Detectioncited by 36
  11. 2020Neural Network Approximations for Calabi-Yau Metricscited by 32
  12. 2021Symmetry-via-Duality: Invariant Neural Network Densities from Parameter-Space Correlatorscited by 28
  13. 2021Building Quantum Field Theories Out of Neuronscited by 27
  14. 2024Clifford-Steerable Convolutional Neural Networkscited by 24
  15. 2023Neural Network Field Theories: Non-Gaussianity, Actions, and Localitycited by 21
  16. 2022Meta-Principled Family of Hyperparameter Scaling Strategiescited by 18
  17. 2024Rigor with Machine Learning from Field Theory to the Poincar'e Conjecturecited by 12
  18. 2022Born-Infeld (BI) for AI: Energy-Conserving Descent (ECD) for Optimizationcited by 10
  19. 2023Improving Energy Conserving Descent for Machine Learning: Theory and Practicecited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 1.

4 papers.

  1. 2025Practical Efficiency of Muon for Pretrainingcited by 74
  2. 2024Conformal Fields from Neural Networkscited by 10
  3. 2025Fermions and Supersymmetry in Neural Network Field Theoriescited by 5
  4. 2026Topological Effects in Neural Network Field Theorycited by 1