# Transferable Physics-Informed Representations via Closed-Form Head Adaptation

arXiv: https://arxiv.org/abs/2604.21761 · HTML: https://mlsanity.com/paper/2604.21761/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 26 references that are in the map.
- First posted on arXiv 23 Apr 2026.

## Builds on (most cited first, 20 of 20)

- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [Revisiting Unreasonable Effectiveness of Data in Deep Learning Era](https://mlsanity.com/paper/1707.02968/) (2017)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next](https://mlsanity.com/paper/2201.05624/) (2022)
- [On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks](https://mlsanity.com/paper/2012.10047/) (2020)
- [Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depth](https://mlsanity.com/paper/2010.15327/) (2020)
- [An Expert's Guide to Training Physics-informed Neural Networks](https://mlsanity.com/paper/2308.08468/) (2023)
- [Transfer learning enhanced physics informed neural network for phase-field modeling of fracture](https://mlsanity.com/paper/1907.02531/) (2019)
- [Local Extreme Learning Machines and Domain Decomposition for Solving Linear and Nonlinear Partial Differential Equations](https://mlsanity.com/paper/2012.02895/) (2020)
- [Learning in Sinusoidal Spaces with Physics-Informed Neural Networks](https://mlsanity.com/paper/2109.09338/) (2021)
- [One-Shot Transfer Learning of Physics-Informed Neural Networks](https://mlsanity.com/paper/2110.11286/) (2021)
- [DPM: A Novel Training Method for Physics-Informed Neural Networks in Extrapolation](https://mlsanity.com/paper/2012.02681/) (2020)
- [Hypernetwork-based Meta-Learning for Low-Rank Physics-Informed Neural Networks](https://mlsanity.com/paper/2310.09528/) (2023)
- [Spline-PINN: Approaching PDEs without Data using Fast, Physics-Informed Hermite-Spline CNNs](https://mlsanity.com/paper/2109.07143/) (2021)
- [A novel meta-learning initialization method for physics-informed neural networks](https://mlsanity.com/paper/2107.10991/) (2021)
- [L-HYDRA: Multi-Head Physics-Informed Neural Networks](https://mlsanity.com/paper/2301.02152/) (2023)
- [PIXEL: Physics-Informed Cell Representations for Fast and Accurate PDE Solvers](https://mlsanity.com/paper/2207.12800/) (2022)
- [Can Transfer Neuroevolution Tractably Solve Your Differential Equations?](https://mlsanity.com/paper/2101.01998/) (2021)
- [Evolutionary Optimization of Physics-Informed Neural Networks: Evo-PINN Frontiers and Opportunities](https://mlsanity.com/paper/2501.06572/) (2025)
- [Evolutionary Optimization of Physics-Informed Neural Networks: Advancing Generalizability by the Baldwin Effect](https://mlsanity.com/paper/2312.03243/) (2023)

## Abstract (arXiv)

Physics-informed neural networks (PINNs) have garnered significant interest for their potential in solving partial differential equations (PDEs) that govern a wide range of physical phenomena. By incorporating physical laws into the learning process, PINN models have demonstrated the ability to learn physical outcomes reasonably well. However, current PINN approaches struggle to predict or solve new PDEs effectively when there is a lack of training examples, indicating they do not generalize well to unseen problem instances. In this paper, we present a transferable learning approach for PINNs premised on a fast Pseudoinverse PINN framework (Pi-PINN). Pi-PINN learns a transferable physics-informed representation in a shared embedding space and enables rapid solving of both known and unknown PDE instances via closed-form head adaptation using a least-squares-optimal pseudoinverse under PDE constraints. We further investigate the synergies between data-driven multi-task learning loss and physics-informed loss, providing insights into the design of more performant PINNs. We demonstrate the effectiveness of Pi-PINN on various PDE problems, including Poisson's equation, Helmholtz equation, and Burgers' equation, achieving fast and accurate physics-informed solutions without requiring any data for unseen instances. Pi-PINN can produce predictions 100-1000 times faster than a typical PINN, while producing predictions with 10-100 times lower relative error than a typical data-driven model even with only two training samples. Overall, our findings highlight the potential of transferable representations with closed-form head adaptation to enhance the efficiency and generalization of PINNs across PDE families and scientific and engineering applications.