# iPINNs: Incremental learning for Physics-informed neural networks

arXiv: https://arxiv.org/abs/2304.04854 · HTML: https://mlsanity.com/paper/2304.04854/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 73 references that are in the map.
- First posted on arXiv 10 Apr 2023.

## Builds on (most cited first, 30 of 36)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning](https://mlsanity.com/paper/1711.05769/) (2017)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [End-to-End Incremental Learning](https://mlsanity.com/paper/1807.09536/) (2018)
- [Physics-informed neural networks (PINNs) for fluid mechanics: A review](https://mlsanity.com/paper/2105.09506/) (2021)
- [A Survey on Deep Transfer Learning](https://mlsanity.com/paper/1808.01974/) (2018)
- [Physics-informed neural networks with hard constraints for inverse design](https://mlsanity.com/paper/2102.04626/) (2021)
- [Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeon](https://mlsanity.com/paper/1705.07565/) (2017)
- [Solving Allen-Cahn and Cahn-Hilliard Equations using the Adaptive Physics Informed Neural Networks](https://mlsanity.com/paper/2007.04542/) (2020)
- [Lorentz Group Equivariant Neural Network for Particle Physics](https://mlsanity.com/paper/2006.04780/) (2020)
- [Transfer learning enhanced physics informed neural network for phase-field modeling of fracture](https://mlsanity.com/paper/1907.02531/) (2019)
- [Class-Incremental Learning by Knowledge Distillation with Adaptive Feature Consolidation](https://mlsanity.com/paper/2204.00895/) (2022)
- [Adversarial Robustness vs Model Compression, or Both?](https://mlsanity.com/paper/1903.12561/) (2019)
- [Sparse DNNs with Improved Adversarial Robustness](https://mlsanity.com/paper/1810.09619/) (2018)
- [Transfer learning based multi-fidelity physics informed deep neural network](https://mlsanity.com/paper/2005.10614/) (2020)
- [SpaceNet: Make Free Space For Continual Learning](https://mlsanity.com/paper/2007.07617/) (2020)
- [Physics-Informed Neural Network for Modelling the Thermochemical Curing Process of Composite-Tool Systems During Manufacture](https://mlsanity.com/paper/2011.13511/) (2020)
- [Transfer learning based physics-informed neural networks for solving inverse problems in engineering structures under different loading scenarios](https://mlsanity.com/paper/2205.07731/) (2022)
- [How Can We Be So Dense? The Benefits of Using Highly Sparse Representations](https://mlsanity.com/paper/1903.11257/) (2019)
- [Neural networks for topology optimization](https://mlsanity.com/paper/1709.09578/) (2017)
- [A novel meta-learning initialization method for physics-informed neural networks](https://mlsanity.com/paper/2107.10991/) (2021)
- [A Metalearning Approach for Physics-Informed Neural Networks (PINNs): Application to Parameterized PDEs](https://mlsanity.com/paper/2110.13361/) (2021)
- [How important are activation functions in regression and classification? A survey, performance comparison, and future directions](https://mlsanity.com/paper/2209.02681/) (2022)
- [L-HYDRA: Multi-Head Physics-Informed Neural Networks](https://mlsanity.com/paper/2301.02152/) (2023)
- [Lagrangian PINNs: A causality-conforming solution to failure modes of physics-informed neural networks](https://mlsanity.com/paper/2205.02902/) (2022)

## Built on by (most cited first, 3 of 3)

- [Neural oscillators for generalization of physics-informed machine learning](https://mlsanity.com/paper/2308.08989/) (2023)
- [Transfer learning for improved generalizability in causal physics-informed neural networks for beam simulations](https://mlsanity.com/paper/2311.00578/) (2023)
- [Physics-Informed DeepONet Coupled with FEM for Convective Transport in Porous Media with Sharp Gaussian Sources](https://mlsanity.com/paper/2508.19847/) (2025)

## Abstract (arXiv)

Physics-informed neural networks (PINNs) have recently become a powerful tool for solving partial differential equations (PDEs). However, finding a set of neural network parameters that lead to fulfilling a PDE can be challenging and non-unique due to the complexity of the loss landscape that needs to be traversed. Although a variety of multi-task learning and transfer learning approaches have been proposed to overcome these issues, there is no incremental training procedure for PINNs that can effectively mitigate such training challenges. We propose incremental PINNs (iPINNs) that can learn multiple tasks (equations) sequentially without additional parameters for new tasks and improve performance for every equation in the sequence. Our approach learns multiple PDEs starting from the simplest one by creating its own subnetwork for each PDE and allowing each subnetwork to overlap with previously learned subnetworks. We demonstrate that previous subnetworks are a good initialization for a new equation if PDEs share similarities. We also show that iPINNs achieve lower prediction error than regular PINNs for two different scenarios: (1) learning a family of equations (e.g., 1-D convection PDE); and (2) learning PDEs resulting from a combination of processes (e.g., 1-D reaction-diffusion PDE). The ability to learn all problems with a single network together with learning more complex PDEs with better generalization than regular PINNs will open new avenues in this field.