# Physics-Informed Neural Networks for High-Frequency and Multi-Scale Problems using Transfer Learning

arXiv: https://arxiv.org/abs/2401.02810 · HTML: https://mlsanity.com/paper/2401.02810/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 56 references that are in the map.
- First posted on arXiv 5 Jan 2024.

## Builds on (most cited first, 22 of 22)

- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [DeepXDE: A deep learning library for solving differential equations](https://mlsanity.com/paper/1907.04502/) (2019)
- [Physics-informed neural networks (PINNs) for fluid mechanics: A review](https://mlsanity.com/paper/2105.09506/) (2021)
- [Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1711.10561/) (2017)
- [What is being transferred in transfer learning?](https://mlsanity.com/paper/2008.11687/) (2020)
- [On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks](https://mlsanity.com/paper/2012.10047/) (2020)
- [Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?](https://mlsanity.com/paper/1706.00712/) (2017)
- [On the Theory of Transfer Learning: The Importance of Task Diversity](https://mlsanity.com/paper/2006.11650/) (2020)
- [Towards Understanding the Spectral Bias of Deep Learning](https://mlsanity.com/paper/1912.01198/) (2019)
- [Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1711.10566/) (2017)
- [Transfer learning enhanced physics informed neural network for phase-field modeling of fracture](https://mlsanity.com/paper/1907.02531/) (2019)
- [Physics informed deep learning for computational elastodynamics without labeled data](https://mlsanity.com/paper/2006.08472/) (2020)
- [Transfer learning based multi-fidelity physics informed deep neural network](https://mlsanity.com/paper/2005.10614/) (2020)
- [The Old and the New: Can Physics-Informed Deep-Learning Replace Traditional Linear Solvers?](https://mlsanity.com/paper/2103.09655/) (2021)
- [Transfer learning based physics-informed neural networks for solving inverse problems in engineering structures under different loading scenarios](https://mlsanity.com/paper/2205.07731/) (2022)
- [Adversarial Training Helps Transfer Learning via Better Representations](https://mlsanity.com/paper/2106.10189/) (2021)
- [On Learning Domain-Invariant Representations for Transfer Learning with Multiple Sources](https://mlsanity.com/paper/2111.13822/) (2021)
- [Transfer Learning Enhanced Full Waveform Inversion](https://mlsanity.com/paper/2302.11259/) (2023)
- [Transfer learning-based physics-informed convolutional neural network for simulating flow in porous media with time-varying controls](https://mlsanity.com/paper/2310.06319/) (2023)

## Built on by (most cited first, 9 of 9)

- [Evolutionary Optimization of Physics-Informed Neural Networks: Evo-PINN Frontiers and Opportunities](https://mlsanity.com/paper/2501.06572/) (2025)
- [Adaptive Physics-informed Neural Networks: A Survey](https://mlsanity.com/paper/2503.18181/) (2025)
- [Iterative Training of Physics-Informed Neural Networks with Fourier-enhanced Features](https://mlsanity.com/paper/2510.19399/) (2025)
- [Improving physics-informed neural network extrapolation via transfer learning and adaptive activation functions](https://mlsanity.com/paper/2507.12659/) (2025)
- [xLSTM-PINN: Memory-Gated Spectral Remodeling for Physics-Informed Learning](https://mlsanity.com/paper/2511.12512/) (2025)
- [NewPINNs: Physics-Informing Neural Networks Using Conventional Solvers for Partial Differential Equations](https://mlsanity.com/paper/2601.17207/) (2026)
- [Estimating Dense-Packed Zone Height in Liquid-Liquid Separation: A Physics-Informed Neural Network Approach](https://mlsanity.com/paper/2601.18399/) (2026)
- [Multi-Scale Separable Fourier Neural Networks for Solving High-Frequency PDEs](https://mlsanity.com/paper/2605.31027/) (2026)
- [When Does Frequency Decomposition Benefit Physics-Informed Neural Networks? A Preliminary Ablation Study](https://mlsanity.com/paper/2608.24940/) (2026)

## Abstract (arXiv)

Physics-informed neural network (PINN) is a data-driven solver for partial and ordinary differential equations(ODEs/PDEs). It provides a unified framework to address both forward and inverse problems. However, the complexity of the objective function often leads to training failures. This issue is particularly prominent when solving high-frequency and multi-scale problems. We proposed using transfer learning to boost the robustness and convergence of training PINN, starting training from low-frequency problems and gradually approaching high-frequency problems. Through two case studies, we discovered that transfer learning can effectively train PINN to approximate solutions from low-frequency problems to high-frequency problems without increasing network parameters. Furthermore, it requires fewer data points and less training time. We elaborately described our training strategy, including optimizer selection, and suggested guidelines for using transfer learning to train neural networks for solving more complex problems.