# REAct: Rational Exponential Activation for Better Learning and Generalization in PINNs

arXiv: https://arxiv.org/abs/2503.02267 · HTML: https://mlsanity.com/paper/2503.02267/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 26 references that are in the map.
- First posted on arXiv 4 Mar 2025.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [When and why PINNs fail to train: A neural tangent kernel perspective](https://mlsanity.com/paper/2007.14527/) (2020)
- [DeepXDE: A deep learning library for solving differential equations](https://mlsanity.com/paper/1907.04502/) (2019)
- [Physics-informed neural networks (PINNs) for fluid mechanics: A review](https://mlsanity.com/paper/2105.09506/) (2021)
- [On the Impact of the Activation Function on Deep Neural Networks Training](https://mlsanity.com/paper/1902.06853/) (2019)
- [Deep Learning of Vortex Induced Vibrations](https://mlsanity.com/paper/1808.08952/) (2018)
- [Learning in Sinusoidal Spaces with Physics-Informed Neural Networks](https://mlsanity.com/paper/2109.09338/) (2021)
- [The Old and the New: Can Physics-Informed Deep-Learning Replace Traditional Linear Solvers?](https://mlsanity.com/paper/2103.09655/) (2021)
- [Learning Specialized Activation Functions for Physics-informed Neural Networks](https://mlsanity.com/paper/2308.04073/) (2023)
- [Neural oscillators for generalization of physics-informed machine learning](https://mlsanity.com/paper/2308.08989/) (2023)

## Abstract (arXiv)

Physics-Informed Neural Networks (PINNs) offer a promising approach to simulating physical systems. Still, their application is limited by optimization challenges, mainly due to the lack of activation functions that generalize well across several physical systems. Existing activation functions often lack such flexibility and generalization power. To address this issue, we introduce Rational Exponential Activation (REAct), a generalized form of tanh consisting of four learnable shape parameters. Experiments show that REAct outperforms many standard and benchmark activations, achieving an MSE three orders of magnitude lower than tanh on heat problems and generalizing well to finer grids and points beyond the training domain. It also excels at function approximation tasks and improves noise rejection in inverse problems, leading to more accurate parameter estimates across varying noise levels.