# A novel meta-learning initialization method for physics-informed neural networks

arXiv: https://arxiv.org/abs/2107.10991 · HTML: https://mlsanity.com/paper/2107.10991/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 69 references that are in the map.
- First posted on arXiv 23 Jul 2021.

## Builds on (most cited first, 26 of 26)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [Meta-Learning with Implicit Gradients](https://mlsanity.com/paper/1909.04630/) (2019)
- [Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data](https://mlsanity.com/paper/1901.06314/) (2019)
- [Probabilistic Model-Agnostic Meta-Learning](https://mlsanity.com/paper/1806.02817/) (2018)
- [hp-VPINNs: Variational Physics-Informed Neural Networks With Domain Decomposition](https://mlsanity.com/paper/2003.05385/) (2020)
- [Why and When Can Deep -- but Not Shallow -- Networks Avoid the Curse of Dimensionality: a Review](https://mlsanity.com/paper/1611.00740/) (2016)
- [PPINN: Parareal Physics-Informed Neural Network for time-dependent PDEs](https://mlsanity.com/paper/1909.10145/) (2019)
- [Transfer learning enhanced physics informed neural network for phase-field modeling of fracture](https://mlsanity.com/paper/1907.02531/) (2019)
- [Parallel Physics-Informed Neural Networks via Domain Decomposition](https://mlsanity.com/paper/2104.10013/) (2021)
- [A fast and accurate physics-informed neural network reduced order model with shallow masked autoencoder](https://mlsanity.com/paper/2009.11990/) (2020)
- [Physics-informed neural network for ultrasound nondestructive quantification of surface breaking cracks](https://mlsanity.com/paper/2005.03596/) (2020)
- [Transfer learning based multi-fidelity physics informed deep neural network](https://mlsanity.com/paper/2005.10614/) (2020)
- [Meta-Learning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithm](https://mlsanity.com/paper/1710.11622/) (2017)
- [Physics-Informed Neural Networks for Multiphysics Data Assimilation with Application to Subsurface Transport](https://mlsanity.com/paper/1912.02968/) (2019)
- [A proof that artificial neural networks overcome the curse of dimensionality in the numerical approximation of Black-Scholes partial differential equations](https://mlsanity.com/paper/1809.02362/) (2018)
- [Physics-guided Convolutional Neural Network (PhyCNN) for Data-driven Seismic Response Modeling](https://mlsanity.com/paper/1909.08118/) (2019)
- [Deep Learning of Turbulent Scalar Mixing](https://mlsanity.com/paper/1811.07095/) (2018)
- [Accelerating Physics-Informed Neural Network Training with Prior Dictionaries](https://mlsanity.com/paper/2004.08151/) (2020)
- [Locally adaptive activation functions with slope recovery term for deep and physics-informed neural networks](https://mlsanity.com/paper/1909.12228/) (2019)
- [Neural networks catching up with finite differences in solving partial differential equations in higher dimensions](https://mlsanity.com/paper/1712.05067/) (2017)

## Built on by (most cited first, 30 of 35)

- [Physics-Informed Machine Learning: A Survey on Problems, Methods and Applications](https://mlsanity.com/paper/2211.08064/) (2022)
- [Multi-Objective Loss Balancing for Physics-Informed Deep Learning](https://mlsanity.com/paper/2110.09813/) (2021)
- [Parameterized Physics-informed Neural Networks for Parameterized PDEs](https://mlsanity.com/paper/2408.09446/) (2024)
- [Meta-Auto-Decoder for Solving Parametric Partial Differential Equations](https://mlsanity.com/paper/2111.08823/) (2021)
- [A Metalearning Approach for Physics-Informed Neural Networks (PINNs): Application to Parameterized PDEs](https://mlsanity.com/paper/2110.13361/) (2021)
- [L-HYDRA: Multi-Head Physics-Informed Neural Networks](https://mlsanity.com/paper/2301.02152/) (2023)
- [Improving physics-informed neural networks with meta-learned optimization](https://mlsanity.com/paper/2303.07127/) (2023)
- [Meta-PDE: Learning to Solve PDEs Quickly Without a Mesh](https://mlsanity.com/paper/2211.01604/) (2022)
- [DeepPhysiNet: Bridging Deep Learning and Atmospheric Physics for Accurate and Continuous Weather Modeling](https://mlsanity.com/paper/2401.04125/) (2024)
- [Evolutionary Optimization of Physics-Informed Neural Networks: Evo-PINN Frontiers and Opportunities](https://mlsanity.com/paper/2501.06572/) (2025)
- [Principled Acceleration of Iterative Numerical Methods Using Machine Learning](https://mlsanity.com/paper/2206.08594/) (2022)
- [Challenges and opportunities for machine learning in multiscale computational modeling](https://mlsanity.com/paper/2303.12261/) (2023)
- [LatentPINNs: Generative physics-informed neural networks via a latent representation learning](https://mlsanity.com/paper/2305.07671/) (2023)
- [Evolutionary Optimization of Physics-Informed Neural Networks: Advancing Generalizability by the Baldwin Effect](https://mlsanity.com/paper/2312.03243/) (2023)
- [Robust Weight Initialization for Tanh Neural Networks with Fixed Point Analysis](https://mlsanity.com/paper/2410.02242/) (2024)
- [Adaptive Physics-informed Neural Networks: A Survey](https://mlsanity.com/paper/2503.18181/) (2025)
- [Learning solutions of parametric Navier-Stokes with physics-informed neural networks](https://mlsanity.com/paper/2402.03153/) (2024)
- [Quantifying Training Difficulty and Accelerating Convergence in Neural Network-Based PDE Solvers](https://mlsanity.com/paper/2410.06308/) (2024)
- [iPINNs: Incremental learning for Physics-informed neural networks](https://mlsanity.com/paper/2304.04854/) (2023)
- [A Sequential Meta-Transfer (SMT) Learning to Combat Complexities of Physics-Informed Neural Networks: Application to Composites Autoclave Processing](https://mlsanity.com/paper/2308.06447/) (2023)
- [Base Models for Parabolic Partial Differential Equations](https://mlsanity.com/paper/2407.12234/) (2024)
- [PIED: Physics-Informed Experimental Design for Inverse Problems](https://mlsanity.com/paper/2503.07070/) (2025)
- [VI-PINNs: Variance-involved Physics-informed Neural Networks for Fast and Accurate Prediction of Partial Differential Equations](https://mlsanity.com/paper/2211.16753/) (2022)
- [A Novel Training Framework for Physics-informed Neural Networks: Towards Real-time Applications in Ultrafast Ultrasound Blood Flow Imaging](https://mlsanity.com/paper/2309.04755/) (2023)
- [Towards Foundation Model for Chemical Reactor Modeling: Meta-Learning with Physics-Informed Adaptation](https://mlsanity.com/paper/2405.11752/) (2024)
- [PinnDE: Physics-Informed Neural Networks for Solving Differential Equations](https://mlsanity.com/paper/2408.10011/) (2024)
- [Parameterized Temporal Extrapolation of PDEs via Disentangled Latent Dynamics Manifold Fusion](https://mlsanity.com/paper/2603.12676/) (2026)
- [Compositional Meta-Learning for Mitigating Task Heterogeneity in Physics-Informed Neural Networks](https://mlsanity.com/paper/2604.26999/) (2026)
- [A physics-driven sensor placement optimization methodology for temperature field reconstruction](https://mlsanity.com/paper/2409.18423/) (2024)
- [Latent Representation Learning of Multi-scale Thermophysics: Application to Dynamics in Shocked Porous Energetic Material](https://mlsanity.com/paper/2506.12996/) (2025)

## Abstract (arXiv)

Physics-informed neural networks (PINNs) have been widely used to solve various scientific computing problems. However, large training costs limit PINNs for some real-time applications. Although some works have been proposed to improve the training efficiency of PINNs, few consider the influence of initialization. To this end, we propose a New Reptile initialization based Physics-Informed Neural Network (NRPINN). The original Reptile algorithm is a meta-learning initialization method based on labeled data. PINNs can be trained with less labeled data or even without any labeled data by adding partial differential equations (PDEs) as a penalty term into the loss function. Inspired by this idea, we propose the new Reptile initialization to sample more tasks from the parameterized PDEs and adapt the penalty term of the loss. The new Reptile initialization can acquire initialization parameters from related tasks by supervised, unsupervised, and semi-supervised learning. Then, PINNs with initialization parameters can efficiently solve PDEs. Besides, the new Reptile initialization can also be used for the variants of PINNs. Finally, we demonstrate and verify the NRPINN considering both forward problems, including solving Poisson, Burgers, and Schr"odinger equations, as well as inverse problems, where unknown parameters in the PDEs are estimated. Experimental results show that the NRPINN training is much faster and achieves higher accuracy than PINNs with other initialization methods.