mlsanityData to 14 Oct 2026

Physical Activation Functions (PAFs): An Approach for More Efficient Induction of Physics into Physics-Informed Neural Networks (PINNs)

arXiv 2205.14630 ↗First posted 29 May 2022cs.LGcited by 10

Not on the map. Why?

Abstract (arXiv)

In recent years, the gap between Deep Learning (DL) methods and analytical or numerical approaches in scientific computing is tried to be filled by the evolution of Physics-Informed Neural Networks (PINNs). However, still, there are many complications in the training of PINNs and optimal interleaving of physical models. Here, we introduced the concept of Physical Activation Functions (PAFs). This concept offers that instead of using general activation functions (AFs) such as ReLU, tanh, and sigmoid for all the neurons, one can use generic AFs that their mathematical expression is inherited from the physical laws of the investigating phenomena. The formula of PAFs may be inspired by the terms in the analytical solution of the problem. We showed that the PAFs can be inspired by any mathematical formula related to the investigating phenomena such as the initial or boundary conditions of the PDE system. We validated the advantages of PAFs for several PDEs including the harmonic oscillations, Burgers, Advection-Convection equation, and the heterogeneous diffusion equations. The main advantage of PAFs was in the more efficient constraining and interleaving of PINNs with the investigating physical phenomena and their underlying mathematical models. This added constraint significantly improved the predictions of PINNs for the testing data that was out-of-training distribution. Furthermore, the application of PAFs reduced the size of the PINNs up to 75% in different cases. Also, the value of loss terms was reduced by 1 to 2 orders of magnitude in some cases which is noteworthy for upgrading the training of the PINNs. The iterations required for finding the optimum values were also significantly reduced. It is concluded that using the PAFs helps in generating PINNs with less complexity and much more validity for longer ranges of prediction.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 23 of its 45 references that are in the map. First posted on arXiv 29 May 2022.

Builds on

23 of 45 references.

References · 23

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 2,018
  3. 2021Characterizing possible failure modes in physics-informed neural networkscited by 643
  4. 2022Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's nextcited by 608
  5. 2019AI Feynman: a Physics-Inspired Method for Symbolic Regressioncited by 548
  6. 2021Physics-informed neural networks (PINNs) for fluid mechanics: A reviewcited by 473
  7. 2021Machine learning accelerated computational fluid dynamicscited by 471
  8. 2020Discovering Symbolic Models from Deep Learning with Inductive Biasescited by 344
  9. 2019Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradientscited by 309
  10. 2017Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equationscited by 161
  11. 2021Parallel Physics-Informed Neural Networks via Domain Decompositioncited by 112
  12. 2020Neural Networks Fail to Learn Periodic Functions and How to Fix Itcited by 100
11 more
  1. 2019Physics-Informed Neural Networks for Power Systemscited by 80
  2. 2019Physics-Informed Neural Networks for Multiphysics Data Assimilation with Application to Subsurface Transportcited by 61
  3. 2021The Old and the New: Can Physics-Informed Deep-Learning Replace Traditional Linear Solvers?cited by 61
  4. 2021Can stable and accurate neural networks be computed? -- On the barriers of deep learning and Smale's 18th problemcited by 53
  5. 2021Deep Kronecker neural networks: A general framework for neural networks with adaptive activation functionscited by 49
  6. 2020DPM: A Novel Training Method for Physics-Informed Neural Networks in Extrapolationcited by 45
  7. 2021Neural Networks with Physics-Informed Architectures and Constraints for Dynamical Systems Modelingcited by 41
  8. 2020Accelerating Physics-Informed Neural Network Training with Prior Dictionariescited by 17
  9. 2019Machine Learning and Big Scientific Datacited by 16
  10. 2021How to Avoid Trivial Solutions in Physics-Informed Neural Networkscited by 15
  11. 2021Physics-informed attention-based neural network for solving non-linear partial differential equationscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 0; 2025: 5; 2026: 2.

10 papers.

  1. 2023Mixed formulation of physics-informed neural networks for thermo-mechanically coupled systems and heterogeneous domainscited by 14
  2. 2025Challenges and Advancements in Modeling Shock Fronts with Physics-Informed Neural Networks: A Review and Benchmarking Studycited by 11
  3. 2023About optimal loss function for training physics-informed neural networks under respecting causalitycited by 6
  4. 2025Enforcing hidden physics in physics-informed neural networkscited by 3
  5. 2025Physics-Informed Neural Networks For Semiconductor Film Deposition: A Reviewcited by 1
  6. 2023One-shot backpropagation for multi-step prediction in physics-based system identification -- EXTENDED VERSIONcited by 0
  7. 2025Plane-Wave Decomposition and Randomised Training; a Novel Path to Generalised PINNs for SHMcited by 0
  8. 2025WellPINN: Accurate Well Representation for Transient Fluid Pressure Diffusion in Subsurface Reservoirs with Physics-Informed Neural Networkscited by 0
  9. 2026A Novel Tensor Product-Based Neural Network for Solving Partial Differential Equationscited by 0
  10. 2026Hyperparameter selection for equation learning with biologically-informed neural networkscited by 0