# Improving physics-informed neural network extrapolation via transfer learning and adaptive activation functions

arXiv: https://arxiv.org/abs/2507.12659 · HTML: https://mlsanity.com/paper/2507.12659/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 26 references that are in the map.
- First posted on arXiv 16 Jul 2025.

## Builds on (most cited first, 13 of 13)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks](https://mlsanity.com/paper/2207.10289/) (2022)
- [Solving Allen-Cahn and Cahn-Hilliard Equations using the Adaptive Physics Informed Neural Networks](https://mlsanity.com/paper/2007.04542/) (2020)
- [Neural Networks Fail to Learn Periodic Functions and How to Fix It](https://mlsanity.com/paper/2006.08195/) (2020)
- [One-Shot Transfer Learning of Physics-Informed Neural Networks](https://mlsanity.com/paper/2110.11286/) (2021)
- [A unified scalable framework for causal sweeping strategies for Physics-Informed Neural Networks (PINNs) and their temporal decompositions](https://mlsanity.com/paper/2302.14227/) (2023)
- [DPM: A Novel Training Method for Physics-Informed Neural Networks in Extrapolation](https://mlsanity.com/paper/2012.02681/) (2020)
- [Investigating molecular transport in the human brain from MRI with physics-informed neural networks](https://mlsanity.com/paper/2205.02592/) (2022)
- [Learning Specialized Activation Functions for Physics-informed Neural Networks](https://mlsanity.com/paper/2308.04073/) (2023)
- [Understanding and Mitigating Extrapolation Failures in Physics-Informed Neural Networks](https://mlsanity.com/paper/2306.09478/) (2023)
- [Physics-Informed Neural Networks for High-Frequency and Multi-Scale Problems using Transfer Learning](https://mlsanity.com/paper/2401.02810/) (2024)
- [Physics-informed Neural Networks with Periodic Activation Functions for Solute Transport in Heterogeneous Porous Media](https://mlsanity.com/paper/2212.08965/) (2022)
- [Transfer learning for improved generalizability in causal physics-informed neural networks for beam simulations](https://mlsanity.com/paper/2311.00578/) (2023)

## Abstract (arXiv)

Physics-Informed Neural Networks (PINNs) are deep learning models that incorporate the governing physical laws of a system into the learning process, making them well-suited for solving complex scientific and engineering problems. Recently, PINNs have gained widespread attention as a powerful framework for combining physical principles with data-driven modeling to improve prediction accuracy. Despite their successes, however, PINNs often exhibit poor extrapolation performance outside the training domain and are highly sensitive to the choice of activation functions (AFs). In this paper, we introduce a transfer learning (TL) method to improve the extrapolation capability of PINNs. Our approach applies transfer learning (TL) within an extended training domain, using only a small number of carefully selected collocation points. Additionally, we propose an adaptive AF that takes the form of a linear combination of standard AFs, which improves both the robustness and accuracy of the model. Through a series of experiments, we demonstrate that our method achieves an average of 40% reduction in relative L2 error and an average of 50% reduction in mean absolute error in the extrapolation domain, all without a significant increase in computational cost. The code is available at https://github.com/LiuzLab/PINN-extrapolation .