mlsanityData to 7 Oct 2026

Les Houches Lectures on Deep Learning at Large & Infinite Width

arXiv 2309.01592 ↗First posted 4 Sep 2023stat.ML cs.AI cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

These lectures, presented at the 2022 Les Houches Summer School on Statistical Physics and Machine Learning, focus on the infinite-width limit and large-width regime of deep neural networks. Topics covered include various statistical and dynamical properties of these networks. In particular, the lecturers discuss properties of random deep neural networks; connections between trained deep neural networks, linear models, kernels, and Gaussian processes that arise in the infinite-width limit; and perturbative and non-perturbative treatments of large but finite-width networks, at initialization and after training.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 17 of its 27 references that are in the map. First posted on arXiv 4 Sep 2023.

Builds on

17 of 27 references.

References · 17

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  2. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 732
  3. 2017Deep Neural Networks as Gaussian Processescited by 617
  4. 2018A Mean Field View of the Landscape of Two-Layers Neural Networkscited by 463
  5. 2018Gaussian Process Behaviour in Wide Deep Neural Networkscited by 314
  6. 2020The large learning rate phase of deep learning: the catapult mechanismcited by 206
  7. 2016Deep Information Propagationcited by 194
  8. 2018Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networkscited by 176
  9. 2021The Principles of Deep Learning Theorycited by 169
  10. 2018Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processescited by 164
  11. 2020Infinite attention: NNGP and NTK for deep attention networkscited by 117
  12. 2019Finite Depth and Width Corrections to the Neural Tangent Kernelcited by 114
5 more
  1. 2019Dynamics of Deep Neural Networks and Neural Tangent Hierarchycited by 96
  2. 2018Which Neural Net Architectures Give Rise To Exploding and Vanishing Gradients?cited by 94
  3. 2019Non-Gaussian processes and neural networks at finite widthscited by 78
  4. 2019Asymptotics of Wide Networks from Feynman Diagramscited by 77
  5. 2018Products of Many Large Random Matrices and Gradients in Deep Neural Networkscited by 49

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Dropout Universality: Scaling Laws and Optimal Scheduling at the Edge-of-Chaoscited by 0