mlsanityData to 7 Oct 2026

Architectural Strategies for the optimization of Physics-Informed Neural Networks

arXiv 2402.02711 ↗First posted 5 Feb 2024cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Physics-informed neural networks (PINNs) offer a promising avenue for tackling both forward and inverse problems in partial differential equations (PDEs) by incorporating deep learning with fundamental physics principles. Despite their remarkable empirical success, PINNs have garnered a reputation for their notorious training challenges across a spectrum of PDEs. In this work, we delve into the intricacies of PINN optimization from a neural architecture perspective. Leveraging the Neural Tangent Kernel (NTK), our study reveals that Gaussian activations surpass several alternate activations when it comes to effectively training PINNs. Building on insights from numerical linear algebra, we introduce a preconditioned neural architecture, showcasing how such tailored architectures enhance the optimization process. Our theoretical findings are substantiated through rigorous validation against established PDEs within the scientific literature.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 29 of its 53 references that are in the map. First posted on arXiv 5 Feb 2024.

Builds on

29 of 53 references.

References · 29

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  2. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  3. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  4. 2018On the Spectral Bias of Neural Networkscited by 1,321
  5. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 784
  6. 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 655
  7. 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 629
  8. 2022Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's nextcited by 603
  9. 2018Lipschitz regularity of deep neural networks: analysis and efficient estimationcited by 413
  10. 2020B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Datacited by 329
  11. 2020On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networkscited by 279
  12. 2023WIRE: Wavelet Implicit Neural Representationscited by 220
17 more
  1. 2019Towards moderate overparameterization: global convergence guarantees for training shallow neural networkscited by 176
  2. 2020Understanding and mitigating gradient pathologies in physics-informed neural networkscited by 134
  3. 2021Beyond Periodicity: Towards a Unifying Framework for Activations in Coordinate-MLPscited by 134
  4. 2020Tight Bounds on the Smallest Eigenvalue of the Neural Tangent Kernel for Deep ReLU Networkscited by 87
  5. 2021Multi-Objective Loss Balancing for Physics-Informed Deep Learningcited by 86
  6. 2022GARF: Gaussian Activated Radiance Fields for High Fidelity Reconstruction and Pose Estimationcited by 86
  7. 2019Measurements of Three-Level Hierarchical Structure in the Outliers in the Spectrum of Deepnet Hessianscited by 68
  8. 2022Overview frequency principle/spectral bias in deep learningcited by 68
  9. 2020Global Convergence of Deep Networks with One Wide Layer Followed by Pyramidal Topologycited by 66
  10. 2019Learning in Modal Space: Solving Time-Dependent Stochastic PDEs Using Physics-Informed Neural Networkscited by 63
  11. 2023PINNsFormer: A Transformer-Based Framework For Physics-Informed Neural Networkscited by 62
  12. 2022Memorization and Optimization in Deep Neural Networks with Minimum Over-parameterizationcited by 36
  13. 2023Improving physics-informed neural networks with meta-learned optimizationcited by 19
  14. 2022Critical Investigation of Failure Modes in Physics-informed Neural Networkscited by 15
  15. 2022Physics-informed Neural Networks with Periodic Activation Functions for Solute Transport in Heterogeneous Porous Mediacited by 8
  16. 2023Curvature-Aware Training for Coordinate Networkscited by 8
  17. 2023Probing optimisation in physics-informed neural networkscited by 1

Built on by

Not cited yet.