# L-HYDRA: Multi-Head Physics-Informed Neural Networks

arXiv: https://arxiv.org/abs/2301.02152 · HTML: https://mlsanity.com/paper/2301.02152/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 66 references that are in the map.
- First posted on arXiv 5 Jan 2023.

## Builds on (most cited first, 30 of 34)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [An Overview of Multi-Task Learning in Deep Neural Networks](https://mlsanity.com/paper/1706.05098/) (2017)
- [Masked Autoregressive Flow for Density Estimation](https://mlsanity.com/paper/1705.07057/) (2017)
- [Improving Variational Inference with Inverse Autoregressive Flow](https://mlsanity.com/paper/1606.04934/) (2016)
- [Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next](https://mlsanity.com/paper/2201.05624/) (2022)
- [Learning the solution operator of parametric partial differential equations with physics-informed DeepOnets](https://mlsanity.com/paper/2103.10974/) (2021)
- [Physics-informed neural networks (PINNs) for fluid mechanics: A review](https://mlsanity.com/paper/2105.09506/) (2021)
- [Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML](https://mlsanity.com/paper/1909.09157/) (2019)
- [B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data](https://mlsanity.com/paper/2003.06097/) (2020)
- [MADE: Masked Autoencoder for Distribution Estimation](https://mlsanity.com/paper/1502.03509/) (2015)
- [Model Reduction and Neural Networks for Parametric PDEs](https://mlsanity.com/paper/2005.03180/) (2020)
- [Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism](https://mlsanity.com/paper/2009.04544/) (2020)
- [Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons](https://mlsanity.com/paper/2201.07766/) (2022)
- [Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations](https://mlsanity.com/paper/1811.02033/) (2018)
- [TensorFlow Distributions](https://mlsanity.com/paper/1711.10604/) (2017)
- [Deep transfer operator learning for partial differential equations under conditional shift](https://mlsanity.com/paper/2204.09810/) (2022)
- [Bayesian Physics-Informed Neural Networks for real-world nonlinear dynamical systems](https://mlsanity.com/paper/2205.08304/) (2022)
- [Bridging Multi-Task Learning and Meta-Learning: Towards Efficient Training and Effective Adaptation](https://mlsanity.com/paper/2106.09017/) (2021)
- [One-Shot Transfer Learning of Physics-Informed Neural Networks](https://mlsanity.com/paper/2110.11286/) (2021)
- [Learning Functional Priors and Posteriors from Data and Physics](https://mlsanity.com/paper/2106.05863/) (2021)
- [A novel meta-learning initialization method for physics-informed neural networks](https://mlsanity.com/paper/2107.10991/) (2021)
- [Normalizing field flows: Solving forward and inverse stochastic differential equations using physics-informed flow models](https://mlsanity.com/paper/2108.12956/) (2021)
- [NeuralUQ: A comprehensive library for uncertainty quantification in neural differential equations and operators](https://mlsanity.com/paper/2208.11866/) (2022)
- [PI-VAE: Physics-Informed Variational Auto-Encoder for stochastic differential equations](https://mlsanity.com/paper/2203.11363/) (2022)
- [A survey of unsupervised learning methods for high-dimensional uncertainty quantification in black-box-type problems](https://mlsanity.com/paper/2202.04648/) (2022)

## Built on by (most cited first, 29 of 29)

- [A comprehensive and FAIR comparison between MLP and KAN representations for differential equations and operator networks](https://mlsanity.com/paper/2406.02917/) (2024)
- [From PINNs to PIKANs: Recent Advances in Physics-Informed Machine Learning](https://mlsanity.com/paper/2410.13228/) (2024)
- [Correcting model misspecification in physics-informed neural networks (PINNs)](https://mlsanity.com/paper/2310.10776/) (2023)
- [Uncertainty quantification for noisy inputs-outputs in physics-informed neural networks and neural operators](https://mlsanity.com/paper/2311.11262/) (2023)
- [Discovering a reaction-diffusion model for Alzheimer's disease by combining PINNs with symbolic regression](https://mlsanity.com/paper/2307.08107/) (2023)
- [Learning and discovering multiple solutions using physics-informed neural networks with random initialization and deep ensemble](https://mlsanity.com/paper/2503.06320/) (2025)
- [Physics-aware Machine Learning Revolutionizes Scientific Paradigm for Machine Learning and Process-based Hydrology](https://mlsanity.com/paper/2310.05227/) (2023)
- [Evolutionary Optimization of Physics-Informed Neural Networks: Evo-PINN Frontiers and Opportunities](https://mlsanity.com/paper/2501.06572/) (2025)
- [Leveraging viscous Hamilton-Jacobi PDEs for uncertainty quantification in scientific machine learning](https://mlsanity.com/paper/2404.08809/) (2024)
- [A Generative Modeling Framework for Inferring Families of Biomechanical Constitutive Laws in Data-Sparse Regimes](https://mlsanity.com/paper/2305.03184/) (2023)
- [Fast meta-solvers for 3D complex-shape scatterers using neural operators trained on a non-scattering problem](https://mlsanity.com/paper/2405.12380/) (2024)
- [NeuroSEM: A hybrid framework for simulating multiphysics problems by coupling PINNs and spectral elements](https://mlsanity.com/paper/2407.21217/) (2024)
- [Efficient PINNs via Multi-Head Unimodular Regularization of the Solutions Space](https://mlsanity.com/paper/2501.12116/) (2025)
- [Leveraging Multi-time Hamilton-Jacobi PDEs for Certain Scientific Machine Learning Problems](https://mlsanity.com/paper/2303.12928/) (2023)
- [LatentPINNs: Generative physics-informed neural networks via a latent representation learning](https://mlsanity.com/paper/2305.07671/) (2023)
- [iPINNs: Incremental learning for Physics-informed neural networks](https://mlsanity.com/paper/2304.04854/) (2023)
- [A Sequential Meta-Transfer (SMT) Learning to Combat Complexities of Physics-Informed Neural Networks: Application to Composites Autoclave Processing](https://mlsanity.com/paper/2308.06447/) (2023)
- [Quantification of total uncertainty in the physics-informed reconstruction of CVSim-6 physiology](https://mlsanity.com/paper/2408.07201/) (2024)
- [Bilevel optimization for learning hyperparameters: Application to solving PDEs and inverse problems with Gaussian processes](https://mlsanity.com/paper/2510.05568/) (2025)
- [HJ-sampler: A Bayesian sampler for inverse problems of a stochastic process by leveraging Hamilton-Jacobi PDEs and score-based generative models](https://mlsanity.com/paper/2409.09614/) (2024)
- [Physics-informed deep learning for infectious disease forecasting](https://mlsanity.com/paper/2501.09298/) (2025)
- [Node Assigned physics-informed neural networks for thermal-hydraulic system simulation: CVH/FL module](https://mlsanity.com/paper/2504.16447/) (2025)
- [Kolmogorov n-Widths for Multitask Physics-Informed Machine Learning (PIML) Methods: Towards Robust Metrics](https://mlsanity.com/paper/2402.11126/) (2024)
- [Scalable physics-informed deep generative model for solving forward and inverse stochastic differential equations](https://mlsanity.com/paper/2503.18012/) (2025)
- [Learning and Transferring Physical Models through Derivatives](https://mlsanity.com/paper/2505.01391/) (2025)
- [LFR-PINO: A Layered Fourier Reduced Physics-Informed Neural Operator for Parametric PDEs](https://mlsanity.com/paper/2506.17582/) (2025)
- [Distribution-Free Robust Predict-Then-Optimize in Function Spaces](https://mlsanity.com/paper/2602.08215/) (2026)
- [An Adaptive Machine Learning Framework for Fluid Flow in Dual-Network Porous Media](https://mlsanity.com/paper/2603.19561/) (2026)
- [Transferable Physics-Informed Representations via Closed-Form Head Adaptation](https://mlsanity.com/paper/2604.21761/) (2026)

## Abstract (arXiv)

We introduce multi-head neural networks (MH-NNs) to physics-informed machine learning, which is a type of neural networks (NNs) with all nonlinear hidden layers as the body and multiple linear output layers as multi-head. Hence, we construct multi-head physics-informed neural networks (MH-PINNs) as a potent tool for multi-task learning (MTL), generative modeling, and few-shot learning for diverse problems in scientific machine learning (SciML). MH-PINNs connect multiple functions/tasks via a shared body as the basis functions as well as a shared distribution for the head. The former is accomplished by solving multiple tasks with MH-PINNs with each head independently corresponding to each task, while the latter by employing normalizing flows (NFs) for density estimate and generative modeling. To this end, our method is a two-stage method, and both stages can be tackled with standard deep learning tools of NNs, enabling easy implementation in practice. MH-PINNs can be used for various purposes, such as approximating stochastic processes, solving multiple tasks synergistically, providing informative prior knowledge for downstream few-shot learning tasks such as meta-learning and transfer learning, learning representative basis functions, and uncertainty quantification. We demonstrate the effectiveness of MH-PINNs in five benchmarks, investigating also the possibility of synergistic learning in regression analysis. We name the open-source code "Lernaean Hydra" (L-HYDRA), since this mythical creature possessed many heads for performing important multiple tasks, as in the proposed method.