Neural Tangent Kernel of Neural Networks with Loss Informed by Differential Operators
Not on the map. Why?
Abstract (arXiv)
Spectral bias is a significant phenomenon in neural network training and can be explained by neural tangent kernel (NTK) theory. In this work, we develop the NTK theory for deep neural networks with physics-informed loss, providing insights into the convergence of NTK during initialization and training, and revealing its explicit structure. We find that, in most cases, the differential operators in the loss function do not induce a faster eigenvalue decay rate and stronger spectral bias. Some experimental results are also presented to verify the theory.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 27 of its 39 references that are in the map. First posted on arXiv 14 Mar 2025.
Builds on
27 of 39 references.
References · 27
- 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
- 2018On the Spectral Bias of Neural Networkscited by 1,321
- 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 784
- 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 732
- 2017DGM: A deep learning algorithm for solving partial differential equationscited by 674
- 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 655
- 2021Characterizing possible failure modes in physics-informed neural networkscited by 632
- 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 629
- 2019On Exact Computation with an Infinitely Wide Neural Netcited by 572
- 2019Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networkscited by 562
- 2017Solving high-dimensional partial differential equations using deep learningcited by 549
- 2020On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networkscited by 279
15 more
- 2019Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Datacited by 261
- 2018Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equationscited by 233
- 2018Training behavior of deep neural network in frequency domaincited by 174
- 2019Towards Understanding the Spectral Bias of Deep Learningcited by 173
- 2019The Convergence Rate of Neural Networks for Learned Functions of Different Frequenciescited by 134
- 2020Deep Equals Shallow for ReLU Networks in Kernel Regimescited by 83
- 2020On the Similarity between the Laplace and Neural Tangent Kernelscited by 76
- 2021Random Neural Networks in the Infinite Width Limit as Gaussian Processescited by 43
- 2024ConFIG: Towards Conflict-free Training of Physics Informed Neural Networkscited by 43
- 2020Regularization Matters: A Nonparametric Perspective on Overparametrized Neural Networkcited by 38
- 2023On the Eigenvalue Decay Rates of a Class of Neural-Network Related Kernel Functions Defined on General Domainscited by 30
- 2023Generalization Ability of Wide Neural Networks on ℝcited by 28
- 2024The Challenges of the Nonlinear Regime for Physics-Informed Neural Networkscited by 21
- 2022On the Spectral Bias of Convolutional Neural Tangent and Gaussian Process Kernelscited by 15
- 2024Generalization Error Curves for Analytic Spectral Algorithms under Power-law Decaycited by 8
Built on by
Citing papers per year: 2026: 3.
3 papers.