A Family of Adaptive Activation Functions for Mitigating Failure Modes in Physics-Informed Neural Networks
Not on the map. Why?
Abstract (arXiv)
Physics-Informed Neural Networks(PINNs) are a powerful and flexible learning framework that has gained significant attention in recent years. It has demonstrated strong performance across a wide range of scientific and engineering problems. In parallel, wavelets have been extensively used as efficient computational tools due to their strong approximation capabilities. Motivated by the common failure modes observed in standard PINNs, this work introduces a novel family of adaptive wavelet-based activation functions. The proposed activation functions significantly improve training stability and expressive power by combining trainable wavelet functions with either trainable or fixed hyperbolic tangent and softplus functions. Five distinct activation functions are developed within the PINN framework and systematically evaluated across four representative classes of partial differential equations (PDEs). Comprehensive comparisons using bar plots demonstrate improved robustness and accuracy compared to traditional activation functions. Furthermore, the proposed approach is validated through direct comparisons with baseline PINNs, transformer-based architectures such as PINNsFormer, and other deep learning models, highlighting its effectiveness and generality.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 57 references that are in the map. First posted on arXiv 18 Mar 2026.
Builds on
25 of 57 references.
References · 25
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2021Characterizing possible failure modes in physics-informed neural networkscited by 643
- 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 634
- 2017Solving high-dimensional partial differential equations using deep learningcited by 556
- 2017Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equationscited by 373
- 2020On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networkscited by 284
- 2018Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equationscited by 235
- 2020Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanismcited by 233
- 2021Activation Functions in Deep Learning: A Comprehensive Survey and Benchmarkcited by 155
- 2018Physics-Informed Generative Adversarial Networks for Stochastic Differential Equationscited by 135
- 2020A survey on modern trainable activation functionscited by 89
- 2023PINNsFormer: A Transformer-Based Framework For Physics-Informed Neural Networkscited by 64
13 more
- 2022Error estimates for physics informed neural networks approximating the Navier-Stokes equationscited by 63
- 2021Learning in Sinusoidal Spaces with Physics-Informed Neural Networkscited by 62
- 2021Quadratic Residual Networks: A New Class of Neural Networks for Solving Forward and Inverse Problems in Physics Involving PDEscited by 43
- 2024Numerical analysis of physics-informed neural networks and related models in physics-informed machine learningcited by 39
- 2020Physics Informed Neural Networks for Simulating Radiative Transfercited by 35
- 2020Review and Comparison of Commonly Used Activation Functions for Deep Neural Networkscited by 21
- 2022Auto-PINN: Understanding and Optimizing Physics-Informed Neural Architecturecited by 14
- 2023Learning Specialized Activation Functions for Physics-informed Neural Networkscited by 14
- 2025Sub-Sequential Physics-Informed Learning with State Space Modelcited by 10
- 2023GaborPINN: Efficient physics informed neural networks using multiplicative filtered networkscited by 7
- 2023Personalized Predictions of Glioblastoma Infiltration: Mathematical Models, Physics-Informed Neural Networks and Multimodal Scanscited by 7
- 2024Learnable Activation Functions in Physics-Informed Neural Networks for Solving Partial Differential Equationscited by 7
- 2020TanhSoft -- a family of activation functions combining Tanh and Softpluscited by 2
Built on by
Not cited yet.