mlsanityData to 13 Oct 2026

The Positivity of the Neural Tangent Kernel

arXiv 2404.12928 ↗First posted 19 Apr 2024cs.LG cs.AI math.PRcited by 13

Not on the map. Why?

Abstract (arXiv)

The Neural Tangent Kernel (NTK) has emerged as a fundamental concept in the study of wide Neural Networks. In particular, it is known that the positivity of the NTK is directly related to the memorization capacity of sufficiently wide networks, i.e., to the possibility of reaching zero loss in training, via gradient descent. Here we will improve on previous works and obtain a sharp result concerning the positivity of the NTK of feedforward networks of any depth. More precisely, we will show that, for any non-polynomial activation function, the NTK is strictly positive definite. Our results are based on a novel characterization of polynomial functions which is of independent interest.

Key facts. Cited by 13 papers in the map (data through 13 Oct 2026). Builds on 16 of its 21 references that are in the map. First posted on arXiv 19 Apr 2024.

Builds on

16 of 21 references.

References · 16

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,610
  2. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 786
  3. 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 658
  4. 2017Deep Neural Networks as Gaussian Processescited by 618
  5. 2018Gradient Descent Finds Global Minima of Deep Neural Networkscited by 573
  6. 2019On Exact Computation with an Infinitely Wide Neural Netcited by 573
  7. 2019Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networkscited by 565
  8. 2019Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networkscited by 222
  9. 2019Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivationcited by 148
  10. 2020Tight Bounds on the Smallest Eigenvalue of the Neural Tangent Kernel for Deep ReLU Networkscited by 87
  11. 2020The Interpolation Phase Transition in Neural Networks: Memorization and Generalization under Lazy Trainingcited by 84
  12. 2020Spectra of the Conjugate Kernel and Neural Tangent Kernel for linear-width neural networkscited by 76
4 more
  1. 2022Memorization and Optimization in Deep Neural Networks with Minimum Over-parameterizationcited by 36
  2. 2023Generalization Ability of Wide Neural Networks on ℝcited by 29
  3. 2019Effect of Activation Functions on the Training of Overparametrized Neural Netscited by 14
  4. 2023Wide neural networks: From non-gaussian random fields at initialization to the NTK geometry of trainingcited by 3

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2024: 2; 2025: 5; 2026: 6.

13 papers.

Sort:

  1. 2024Simplicity bias and optimization threshold in two-layer ReLU networkscited by 11
  2. 2025Convergence of Stochastic Gradient Methods for Wide Two-Layer Physics-Informed Neural Networks for the Poisson Equationcited by 2
  3. 2025Trace Regularity PINNs: Enforcing H^frac12(partial Omega) for Boundary Datacited by 2
  4. 2024Thermodynamic limit in learning period threecited by 1
  5. 2025Quantitative convergence of trained single layer neural networks to Gaussian processescited by 1
  6. 2025Querying Kernel Methods Suffices for Reconstructing their Training Datacited by 0
  7. 2025Posterior Bayesian Neural Networks with Dependent Weightscited by 0
  8. 2026Convergence Analysis of Newton's Method for Neural Networks in the Overparameterized Limitcited by 0
  9. 2026The Neural Tangent Kernel for Classificationcited by 0
  10. 2026Optimal Non-Asymptotic Edgeworth Expansions for Multivariate Neural Network Outputscited by 0
  11. 2026The Differential Neural Tangent Kernel and Its Positivitycited by 0
  12. 2026Dimension-Free Rank Lifting from Random Hyperplane Arrangementscited by 0
1 more
  1. 2026Global Communication or Graph-Specific Memory?cited by 0