Data to 14 Oct 2026

Error convergence and engineering-guided hyperparameter search of PINNs: towards optimized I-FENN performance

arXiv 2303.03918 ↗First posted 3 Mar 2023cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

In our recently proposed Integrated Finite Element Neural Network (I-FENN) framework (Pantidis and Mobasher, 2023) we showcased how PINNs can be deployed on a finite element-level basis to swiftly approximate a state variable of interest, and we applied it in the context of non-local gradient-enhanced damage mechanics. In this paper, we enhance the rigour and performance of I-FENN by focusing on two crucial aspects of its PINN component: a) the error convergence analysis and b) the hyperparameter-performance relationship. Guided by the available theoretical formulations in the field, we introduce a systematic numerical approach based on a novel set of holistic performance metrics to answer both objectives. In the first objective, we explore in detail the convergence of the PINN training error and the global error against the network size and the training sample size. We demonstrate a consistent converging behavior of the two error types for any investigated combination of network complexity, dataset size and choice of hyperparameters, which empirically proves the conformance of the PINN setup and implementation to the available convergence theories. In the second objective, we establish an a-priori knowledge of the hyperparameters which favor higher predictive accuracy, lower computational effort, and the least chances of arriving at trivial solutions. The analysis leads to several outcomes that contribute to the better performance of I-FENN, and fills a long-standing gap in the PINN literature with regards to the numerical convergence of the network errors while accounting for commonly used optimizers (Adam and L-BFGS). The proposed analysis method can be directly extended to other ML applications in science and engineering. The code and data utilized in the analysis are posted publicly to aid the reproduction and extension of this research.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 38 of its 83 references that are in the map. First posted on arXiv 3 Mar 2023.

Builds on

38 of 83 references.

References · 38

2015Deep Residual Learning for Image Recognitioncited by 52,096
2014Adam: A Method for Stochastic Optimizationcited by 37,240
2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 2,018
2015Automatic differentiation in machine learning: a surveycited by 855
2021Characterizing possible failure modes in physics-informed neural networkscited by 643
2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 634
2022Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's nextcited by 608
2016Error bounds for approximations with deep ReLU networkscited by 574
2021Learning the solution operator of parametric partial differential equations with physics-informed DeepOnetscited by 540
2017Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equationscited by 373
2020B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Datacited by 335
2017The Expressive Power of Neural Networks: A View from the Widthcited by 320

26 more

2016On the Expressive Power of Deep Neural Networkscited by 306
2021Physics-informed neural networks with hard constraints for inverse designcited by 280
2020On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practicecited by 182
2018Optimal approximation of continuous functions by very deep ReLU networkscited by 166
2019Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivationcited by 149
2019Variational Physics-Informed Neural Networks For Solving Partial Differential Equationscited by 148
2019Deep Network Approximation Characterized by Number of Neuronscited by 122
2017Universal Function Approximation by Deep Neural Nets with Bounded Width and ReLU Activationscited by 104
2020Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating a class of inverse problems for PDEscited by 102
2018Which Neural Net Architectures Give Rise To Exploding and Vanishing Gradients?cited by 94
2021On the approximation of functions by tanh neural networkscited by 81
2020Physics informed deep learning for computational elastodynamics without labeled datacited by 80
2016Deep vs. shallow networks : An approximation theory perspectivecited by 63
2021The Modern Mathematics of Deep Learningcited by 63
2022Error estimates for physics informed neural networks approximating the Navier-Stokes equationscited by 63
2021The Old and the New: Can Physics-Informed Deep-Learning Replace Traditional Linear Solvers?cited by 61
2016Depth-Width Tradeoffs in Approximating Natural Functions with Neural Networkscited by 59
2021Physics informed neural networks for continuum micromechanicscited by 46
2020Physics Informed Neural Networks for Simulating Radiative Transfercited by 35
2021Data vs. Physics: The Apparent Pareto Front of Physics-Informed Neural Networkscited by 27
2021Conditional physics informed neural networkscited by 23
2022Hyper-parameter tuning of physics-informed neural networks: Application to Helmholtz problemscited by 19
2022Auto-PINN: Understanding and Optimizing Physics-Informed Neural Architecturecited by 14
2022Integrated Finite Element Neural Network (I-FENN) for non-local continuum damage mechanicscited by 13
2018Effect of Depth and Width on Local Minima in Deep Learningcited by 12
2021Lipschitz regularization for fracture: the Lip-field approachcited by 1

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 3; 2025: 1.

5 papers.