mlsanityData to 14 Oct 2026

The Differential Neural Tangent Kernel and Its Positivity

arXiv 2607.10200 ↗First posted 11 Jul 2026cs.LG cs.NA math.NAcited by 0

Not on the map. Why?

Abstract (arXiv)

The Neural Tangent Kernel (NTK) is one powerful tool for analyzing the training dynamics of neural networks in the over-parameterized regime. Recently, the theoretical framework has been extended to physics-informed neural networks (PINNs) for solving linear PDEs, one highly popular class of neural PDE solvers. In the analysis, the positivity of the associated NTK plays a fundamental role. However, establishing the positivity of the NTK for PINNs is highly challenging, due to the presence of multiple differential operators. In this work, we propose a new theoretical framework, called Differential Neural Tangent Kernel (DNTK), for analyzing PINNs through the lens of the NTK, and establish the positivity of the infinite width DNTK for both shallow and deep neural networks for a wide class of activation functions, including RePU and smooth but non-polynomial activations, for all linear differential operators. These theoretical results lay the foundation for the analysis of gradient type algorithms for training PINNs.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 30 of its 51 references that are in the map. First posted on arXiv 11 Jul 2026.

Builds on

30 of 51 references.

References · 30

  1. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  2. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  3. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 787
  4. 2017DGM: A deep learning algorithm for solving partial differential equationscited by 678
  5. 2018On Lazy Training in Differentiable Programmingcited by 675
  6. 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 659
  7. 2018Gradient Descent Finds Global Minima of Deep Neural Networkscited by 574
  8. 2019On Exact Computation with an Infinitely Wide Neural Netcited by 573
  9. 2017Solving high-dimensional partial differential equations using deep learningcited by 556
  10. 2020On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networkscited by 284
  11. 2019Towards Understanding the Spectral Bias of Deep Learningcited by 174
  12. 2019Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivationcited by 149
18 more
  1. 2020Tight Bounds on the Smallest Eigenvalue of the Neural Tangent Kernel for Deep ReLU Networkscited by 87
  2. 2020On the Similarity between the Laplace and Neural Tangent Kernelscited by 76
  3. 2024From PINNs to PIKANs: Recent Advances in Physics-Informed Machine Learningcited by 73
  4. 2021Physics-informed neural networks for solving Reynolds-averaged Navierunicodex2013Stokes equationscited by 68
  5. 2020Two-Layer Neural Networks for Partial Differential Equations: Optimization and Generalization Theorycited by 41
  6. 2024Numerical analysis of physics-informed neural networks and related models in physics-informed machine learningcited by 39
  7. 2022Memorization and Optimization in Deep Neural Networks with Minimum Over-parameterizationcited by 36
  8. 2022Deep Learning Methods for Partial Differential Equations and Related Parameter Identification Problemscited by 34
  9. 2016Diverse Neural Network Learns True Target Functionscited by 33
  10. 2023On the Eigenvalue Decay Rates of a Class of Neural-Network Related Kernel Functions Defined on General Domainscited by 31
  11. 2019Effect of Activation Functions on the Training of Overparametrized Neural Netscited by 14
  12. 2024The Positivity of the Neural Tangent Kernelcited by 13
  13. 2024Bounds for the smallest eigenvalue of the NTK for arbitrary spherical data of arbitrary dimensioncited by 13
  14. 2024Convergence Analysis of Natural Gradient Descent for Over-parameterized Physics-Informed Neural Networkscited by 5
  15. 2025Neural Tangent Kernel of Neural Networks with Loss Informed by Differential Operatorscited by 4
  16. 2024Convergence of Implicit Gradient Descent for Training Two-Layer Physics-Informed Neural Networkscited by 3
  17. 2023Conductivity Imaging from Internal Measurements with Mixed Least-Squares Deep Neural Networkscited by 2
  18. 2025Convergence of Stochastic Gradient Methods for Wide Two-Layer Physics-Informed Neural Networks for the Poisson Equationcited by 2

Built on by

Not cited yet.