# A multifidelity approach to continual learning for physical systems

arXiv: https://arxiv.org/abs/2304.03894 · HTML: https://mlsanity.com/paper/2304.03894/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 72 references that are in the map.
- First posted on arXiv 8 Apr 2023.

## Builds on (most cited first, 30 of 32)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Memory Aware Synapses: Learning what (not) to forget](https://mlsanity.com/paper/1711.09601/) (2017)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [When and why PINNs fail to train: A neural tangent kernel perspective](https://mlsanity.com/paper/2007.14527/) (2020)
- [Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next](https://mlsanity.com/paper/2201.05624/) (2022)
- [Physics-informed neural networks (PINNs) for fluid mechanics: A review](https://mlsanity.com/paper/2105.09506/) (2021)
- [AdapterHub: A Framework for Adapting Transformers](https://mlsanity.com/paper/2007.07779/) (2020)
- [BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning](https://mlsanity.com/paper/2002.06715/) (2020)
- [Meta Networks](https://mlsanity.com/paper/1703.00837/) (2017)
- [A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks](https://mlsanity.com/paper/2207.10289/) (2022)
- [Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism](https://mlsanity.com/paper/2009.04544/) (2020)
- [Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines](https://mlsanity.com/paper/1810.12488/) (2018)
- [Solving Allen-Cahn and Cahn-Hilliard Equations using the Adaptive Physics Informed Neural Networks](https://mlsanity.com/paper/2007.04542/) (2020)
- [PPINN: Parareal Physics-Informed Neural Network for time-dependent PDEs](https://mlsanity.com/paper/1909.10145/) (2019)
- [Efficient training of physics-informed neural networks via importance sampling](https://mlsanity.com/paper/2104.12325/) (2021)
- [Multifidelity deep neural operators for efficient learning of partial differential equations with application to fast inverse design of nanoscale heat transport](https://mlsanity.com/paper/2204.06684/) (2022)
- [Long-time integration of parametric evolution equations with physics-informed DeepONets](https://mlsanity.com/paper/2106.05384/) (2021)
- [Physics-Informed Neural Networks for Power Systems](https://mlsanity.com/paper/1911.03737/) (2019)
- [Rehearsal revealed: The limits and merits of revisiting samples in continual learning](https://mlsanity.com/paper/2104.07446/) (2021)
- [Failure-informed adaptive sampling for PINNs](https://mlsanity.com/paper/2210.00279/) (2022)
- [Multifidelity Deep Operator Networks For Data-Driven and Physics-Informed Problems](https://mlsanity.com/paper/2204.09157/) (2022)
- [DAE-PINN: A Physics-Informed Neural Network Model for Simulating Differential-Algebraic Equations with Application to Power Networks](https://mlsanity.com/paper/2109.04304/) (2021)
- [Galerkin Neural Networks: A Framework for Approximating Variational Equations with Error Control](https://mlsanity.com/paper/2105.14094/) (2021)
- [On the Role of Fixed Points of Dynamical Systems in Training Physics-Informed Neural Networks](https://mlsanity.com/paper/2203.13648/) (2022)
- [Importance Driven Continual Learning for Segmentation Across Domains](https://mlsanity.com/paper/2005.00079/) (2020)
- [Recipes for when Physics Fails: Recovering Robust Learning of Physics Informed Neural Networks](https://mlsanity.com/paper/2110.13330/) (2021)
- [Applying Machine Learning to Study Fluid Mechanics](https://mlsanity.com/paper/2110.02083/) (2021)
- [A Multifidelity deep operator network approach to closure for multiscale systems](https://mlsanity.com/paper/2303.08893/) (2023)

## Built on by (most cited first, 13 of 13)

- [Finite basis Kolmogorov-Arnold networks: domain decomposition for data-driven and physics-informed problems](https://mlsanity.com/paper/2406.19662/) (2024)
- [Stacked networks improve physics-informed training: applications to neural networks and deep operator networks](https://mlsanity.com/paper/2311.06483/) (2023)
- [Self-adaptive weights based on balanced residual decay rate for physics-informed neural networks and deep operator networks](https://mlsanity.com/paper/2407.01613/) (2024)
- [SPIKANs: Separable Physics-Informed Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2411.06286/) (2024)
- [Multifidelity Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2410.14764/) (2024)
- [Transfer Learning on Multi-Dimensional Data: A Novel Approach to Neural Network-Based Surrogate Modeling](https://mlsanity.com/paper/2410.12241/) (2024)
- [iPINNER: An Iterative Physics-Informed Neural Network with Ensemble Kalman Filter](https://mlsanity.com/paper/2506.00731/) (2025)
- [Physics-Informed DeepONet Coupled with FEM for Convective Transport in Porous Media with Sharp Gaussian Sources](https://mlsanity.com/paper/2508.19847/) (2025)
- [SLE-FNO: Single-Layer Extensions for Task-Agnostic Continual Learning in Fourier Neural Operators](https://mlsanity.com/paper/2603.20410/) (2026)
- [Bridging quantum and classical computing for partial differential equations through multifidelity machine learning](https://mlsanity.com/paper/2512.05241/) (2025)
- [Domain-Decomposed Graph Neural Network Surrogate Modeling for Ice Sheets](https://mlsanity.com/paper/2512.01888/) (2025)
- [Multifidelity-Augmented Gaussian Process Inputs for Surrogate Modeling from Scarce Data](https://mlsanity.com/paper/2603.22050/) (2026)
- [From Classification to Regression: Using a Fruitfly to Solve Equations](https://mlsanity.com/paper/2607.27196/) (2026)

## Abstract (arXiv)

We introduce a novel continual learning method based on multifidelity deep neural networks. This method learns the correlation between the output of previously trained models and the desired output of the model on the current training dataset, limiting catastrophic forgetting. On its own the multifidelity continual learning method shows robust results that limit forgetting across several datasets. Additionally, we show that the multifidelity method can be combined with existing continual learning methods, including replay and memory aware synapses, to further limit catastrophic forgetting. The proposed continual learning method is especially suited for physical problems where the data satisfy the same physical laws on each domain, or for physics-informed neural networks, because in these cases we expect there to be a strong correlation between the output of the previous model and the model on the current training domain.