mlsanityData to 7 Oct 2026

Spectral-Bias and Kernel-Task Alignment in Physically Informed Neural Networks

arXiv 2307.06362 ↗First posted 12 Jul 2023stat.ML cond-mat.dis-nn cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Physically informed neural networks (PINNs) are a promising emerging method for solving differential equations. As in many other deep learning approaches, the choice of PINN design and training protocol requires careful craftsmanship. Here, we suggest a comprehensive theoretical framework that sheds light on this important problem. Leveraging an equivalence between infinitely over-parameterized neural networks and Gaussian process regression (GPR), we derive an integro-differential equation that governs PINN prediction in the large data-set limit -- the neurally-informed equation. This equation augments the original one by a kernel term reflecting architecture choices and allows quantifying implicit bias induced by the network via a spectral decomposition of the source term in the original differential equation.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 36 of its 59 references that are in the map. First posted on arXiv 12 Jul 2023.

Builds on

36 of 59 references.

References · 36

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  3. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  4. 2021Characterizing possible failure modes in physics-informed neural networkscited by 632
  5. 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 629
  6. 2017Deep Neural Networks as Gaussian Processescited by 617
  7. 2019DeepXDE: A deep learning library for solving differential equationscited by 617
  8. 2022Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's nextcited by 603
  9. 2021Physics-informed neural networks (PINNs) for fluid mechanics: A reviewcited by 470
  10. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 307
  11. 2017Stochastic Gradient Descent as Approximate Bayesian Inferencecited by 298
  12. 2020On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networkscited by 279
24 more
  1. 2020Spectral Bias and Task-Model Alignment Explain Generalization in Kernel Regression and Infinitely Wide Neural Networkscited by 166
  2. 2018Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processescited by 164
  3. 2017Machine Learning of Linear Differential Equations using Gaussian Processescited by 139
  4. 2018Deep Convolutional Networks as shallow Gaussian Processescited by 137
  5. 2020Infinite attention: NNGP and NTK for deep attention networkscited by 117
  6. 2020Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating PDEscited by 115
  7. 2022A Solvable Model of Neural Scaling Lawscited by 101
  8. 2021Generalization Error Rates in Kernel Regression: The Crossover from the Noiseless to Noisy Regimecited by 95
  9. 2022Physics-Informed Machine Learning: A Survey on Problems, Methods and Applicationscited by 88
  10. 2021Improved architectures and training algorithms for deep operator networkscited by 85
  11. 2020Statistical Mechanics of Deep Linear Neural Networks: The Back-Propagating Kernel Renormalizationcited by 71
  12. 2023Can Physics-Informed Neural Networks beat the Finite Element Method?cited by 68
  13. 2022Generic bounds on the approximation error for physics-informed (and) operator learningcited by 65
  14. 2022Error estimates for physics informed neural networks approximating the Navier-Stokes equationscited by 63
  15. 2021The Old and the New: Can Physics-Informed Deep-Learning Replace Traditional Linear Solvers?cited by 61
  16. 2021Separation of Scales and a Thermodynamic Description of Feature Learning in Some CNNscited by 53
  17. 2020Predicting the outputs of finite deep neural networks trained with noisy gradientscited by 31
  18. 2022wPINNs: Weak Physics informed neural networks for approximating entropy solutions of hyperbolic conservation lawscited by 29
  19. 2022Lagrangian PINNs: A causality-conforming solution to failure modes of physics-informed neural networkscited by 26
  20. 2022Partial Differential Equations Meet Deep Neural Networks: A Surveycited by 25
  21. 2022Physics-Informed Neural Networks with Adaptive Localized Artificial Viscositycited by 21
  22. 2022Bayesian Interpolation with Deep Linear Networkscited by 21
  23. 2022State-of-the-Art Review of Design of Experiments for Physics-Informed Deep Learningcited by 12
  24. 2022Calibrating constitutive models with full-field data via physics informed neural networkscited by 12

Built on by

Not cited yet.