mlsanityData to 7 Oct 2026

Self-scalable Tanh (Stan): Faster Convergence and Better Generalization in Physics-informed Neural Networks

arXiv 2204.12589 ↗First posted 26 Apr 2022cs.LG cs.NEcited by 5

Not on the map. Why?

Abstract (arXiv)

Physics-informed Neural Networks (PINNs) are gaining attention in the engineering and scientific literature for solving a range of differential equations with applications in weather modeling, healthcare, manufacturing, etc. Poor scalability is one of the barriers to utilizing PINNs for many real-world problems. To address this, a Self-scalable tanh (Stan) activation function is proposed for the PINNs. The proposed Stan function is smooth, non-saturating, and has a trainable parameter. During training, it can allow easy flow of gradients to compute the required derivatives and also enable systematic scaling of the input-output mapping. It is shown theoretically that the PINNs with the proposed Stan function have no spurious stationary points when using gradient descent algorithms. The proposed Stan is tested on a number of numerical studies involving general regression problems. It is subsequently used for solving multiple forward problems, which involve second-order derivatives and multiple dimensions, and an inverse problem where the thermal diffusivity of a rod is predicted with heat conduction data. These case studies establish empirically that the Stan activation function can achieve better training and more accurate predictions than the existing activation functions in the literature.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 12 of its 50 references that are in the map. First posted on arXiv 26 Apr 2022.

Builds on

12 of 50 references.

References · 12

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015Automatic differentiation in machine learning: a surveycited by 846
  3. 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 629
  4. 2017Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equationscited by 208
  5. 2021Finite Basis Physics-Informed Neural Networks (FBPINNs): a scalable domain decomposition approach for solving differential equationscited by 142
  6. 2017Spurious Local Minima are Common in Two-Layer ReLU Neural Networkscited by 104
  7. 2019On the Impact of the Activation Function on Deep Neural Networks Trainingcited by 101
  8. 2020Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating a class of inverse problems for PDEscited by 101
  9. 2016Noisy Activation Functionscited by 53
  10. 2020A Physics-Informed Machine Learning Approach for Solving Heat Transfer Equation in Advanced Manufacturing and Engineering Applicationscited by 53
  11. 2020Machine learning for metal additive manufacturing: Predicting temperature and melt pool fluid dynamics using physics-informed neural networkscited by 42
  12. 2021WGAN with an Infinitely Wide Generator Has No Spurious Stationary Pointscited by 1

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 0; 2024: 3; 2025: 1.

5 papers.

  1. 2024Adaptive Training of Grid-Dependent Physics-Informed Kolmogorov-Arnold Networkscited by 37
  2. 2024Recent Advances on Machine Learning for Computational Fluid Dynamics: A Surveycited by 27
  3. 2022Modular machine learning-based elastoplasticity: generalization in the context of limited datacited by 16
  4. 2024Extremization to Fine Tune Physics Informed Neural Networks for Solving Boundary Value Problemscited by 0
  5. 2025Physics Informed Neural Networks for design optimisation of diamond particle detectors for charged particle fast-tracking at high luminosity hadron colliderscited by 0