# GPT-PINN: Generative Pre-Trained Physics-Informed Neural Networks toward non-intrusive Meta-learning of parametric PDEs

arXiv: https://arxiv.org/abs/2303.14878 · HTML: https://mlsanity.com/paper/2303.14878/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 51 references that are in the map.
- First posted on arXiv 27 Mar 2023.

## Builds on (most cited first, 18 of 18)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems](https://mlsanity.com/paper/1710.00211/) (2017)
- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1801.06637/) (2018)
- [Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism](https://mlsanity.com/paper/2009.04544/) (2020)
- [Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1708.00588/) (2017)
- [Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations](https://mlsanity.com/paper/1706.04702/) (2017)
- [Respecting causality is all you need for training physics-informed neural networks](https://mlsanity.com/paper/2203.07404/) (2022)
- [Solving Allen-Cahn and Cahn-Hilliard Equations using the Adaptive Physics Informed Neural Networks](https://mlsanity.com/paper/2007.04542/) (2020)
- [Meta-learning PINN loss functions](https://mlsanity.com/paper/2107.05544/) (2021)
- [Solving for high dimensional committor functions using artificial neural networks](https://mlsanity.com/paper/1802.10275/) (2018)
- [Transferring Knowledge across Learning Processes](https://mlsanity.com/paper/1812.01054/) (2018)
- [Meta-PDE: Learning to Solve PDEs Quickly Without a Mesh](https://mlsanity.com/paper/2211.01604/) (2022)
- [MetaNO: How to Transfer Your Knowledge on Learning Hidden Physics](https://mlsanity.com/paper/2301.12095/) (2023)
- [On the approximation of rough functions with deep neural networks](https://mlsanity.com/paper/1912.06732/) (2019)

## Built on by (most cited first, 20 of 20)

- [Spectral-Refiner: Accurate Fine-Tuning of Spatiotemporal Fourier Neural Operator for Turbulent Flows](https://mlsanity.com/paper/2405.17211/) (2024)
- [Evolutionary Optimization of Physics-Informed Neural Networks: Evo-PINN Frontiers and Opportunities](https://mlsanity.com/paper/2501.06572/) (2025)
- [AcademicGPT: Empowering Academic Research](https://mlsanity.com/paper/2311.12315/) (2023)
- [LatentPINNs: Generative physics-informed neural networks via a latent representation learning](https://mlsanity.com/paper/2305.07671/) (2023)
- [Evolutionary Optimization of Physics-Informed Neural Networks: Advancing Generalizability by the Baldwin Effect](https://mlsanity.com/paper/2312.03243/) (2023)
- [TGPT-PINN: Nonlinear model reduction with transformed GPT-PINNs](https://mlsanity.com/paper/2403.03459/) (2024)
- [Adaptive Physics-informed Neural Networks: A Survey](https://mlsanity.com/paper/2503.18181/) (2025)
- [Resolution invariant deep operator network for PDEs with complex geometries](https://mlsanity.com/paper/2402.00825/) (2024)
- [Meta-learning Loss Functions of Parametric Partial Differential Equations Using Physics-Informed Neural Networks](https://mlsanity.com/paper/2412.00225/) (2024)
- [GITO: Graph-Informed Transformer Operator for Learning Complex Partial Differential Equations](https://mlsanity.com/paper/2506.13906/) (2025)
- [S2GPT-PINNs: Sparse and Small models for PDEs](https://mlsanity.com/paper/2506.15687/) (2025)
- [ReBaNO: Reduced Basis Neural Operator Mitigating Generalization Gaps and Achieving Discretization Invariance](https://mlsanity.com/paper/2509.09611/) (2025)
- [BO-SA-PINNs: Self-adaptive physics-informed neural networks based on Bayesian optimization for automatically designing PDE solvers](https://mlsanity.com/paper/2504.09804/) (2025)
- [Position: Certifiable State Integrity Should Be Built from Local Validity, Not Global Scale](https://mlsanity.com/paper/2601.21249/) (2026)
- [Fiber Transmission Model with Parameterized Inputs based on GPT-PINN Neural Network](https://mlsanity.com/paper/2408.09947/) (2024)
- [Principle Driven Parameterized Fiber Model based on GPT-PINN Neural Network](https://mlsanity.com/paper/2408.09951/) (2024)
- [Enhanced physics-informed neural networks (PINNs) for high-order power grid dynamics](https://mlsanity.com/paper/2410.07527/) (2024)
- [IP-Basis PINNs: Efficient Multi-Query Inverse Parameter Estimation](https://mlsanity.com/paper/2509.07245/) (2025)
- [Architecture Shapes Transfer Specificity in Implicit Neural Representations](https://mlsanity.com/paper/2606.06827/) (2026)
- [Does Text Steer Neural PDE Surrogates? A Controlled Diagnostic with OperatorCLIP](https://mlsanity.com/paper/2609.38517/) (2026)

## Abstract (arXiv)

Physics-Informed Neural Network (PINN) has proven itself a powerful tool to obtain the numerical solutions of nonlinear partial differential equations (PDEs) leveraging the expressivity of deep neural networks and the computing power of modern heterogeneous hardware. However, its training is still time-consuming, especially in the multi-query and real-time simulation settings, and its parameterization often overly excessive. In this paper, we propose the Generative Pre-Trained PINN (GPT-PINN) to mitigate both challenges in the setting of parametric PDEs. GPT-PINN represents a brand-new meta-learning paradigm for parametric systems. As a network of networks, its outer-/meta-network is hyper-reduced with only one hidden layer having significantly reduced number of neurons. Moreover, its activation function at each hidden neuron is a (full) PINN pre-trained at a judiciously selected system configuration. The meta-network adaptively ``learns'' the parametric dependence of the system and ``grows'' this hidden layer one neuron at a time. In the end, by encompassing a very small number of networks trained at this set of adaptively-selected parameter values, the meta-network is capable of generating surrogate solutions for the parametric system across the entire parameter domain accurately and efficiently.