mlsanityData to 7 Oct 2026

On neural network kernels and the storage capacity problem

arXiv 2201.04669 ↗First posted 12 Jan 2022cond-mat.dis-nn cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

In this short note, we reify the connection between work on the storage capacity problem in wide two-layer treelike neural networks and the rapidly-growing body of literature on kernel limits of wide neural networks. Concretely, we observe that the "effective order parameter" studied in the statistical mechanics literature is exactly equivalent to the infinite-width Neural Network Gaussian Process Kernel. This correspondence connects the expressivity and trainability of wide two-layer neural networks.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 9 of its 21 references that are in the map. First posted on arXiv 12 Jan 2022.

Builds on

9 of 21 references.

References · 9

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  2. 2017Deep Neural Networks as Gaussian Processescited by 617
  3. 2018Gaussian Process Behaviour in Wide Deep Neural Networkscited by 314
  4. 2016Exponential expressivity in deep neural networks through transient chaoscited by 260
  5. 2016Toward Deeper Understanding of Neural Networks: The Power of Initialization and a Dual View on Expressivitycited by 138
  6. 2020Exact posterior distributions of wide Bayesian neural networkscited by 29
  7. 2019Properties of the geometry of solutions and capacity of multi-layer neural networks with Rectified Linear Units activationscited by 22
  8. 2019Effect of Activation Functions on the Training of Overparametrized Neural Netscited by 14
  9. 2020Activation function dependence of the storage capacity of treelike neural networkscited by 10

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 1.

2 papers.

  1. 2023Long Sequence Hopfield Memorycited by 15
  2. 2024Exact full-RSB SAT/UNSAT transition in infinitely wide two-layer neural networkscited by 6