mlsanityData to 14 Oct 2026

A multifidelity approach to continual learning for physical systems

arXiv 2304.03894 ↗First posted 8 Apr 2023math.NA cs.LG cs.NAcited by 13

Not on the map. Why?

Abstract (arXiv)

We introduce a novel continual learning method based on multifidelity deep neural networks. This method learns the correlation between the output of previously trained models and the desired output of the model on the current training dataset, limiting catastrophic forgetting. On its own the multifidelity continual learning method shows robust results that limit forgetting across several datasets. Additionally, we show that the multifidelity method can be combined with existing continual learning methods, including replay and memory aware synapses, to further limit catastrophic forgetting. The proposed continual learning method is especially suited for physical problems where the data satisfy the same physical laws on each domain, or for physics-informed neural networks, because in these cases we expect there to be a strong correlation between the output of the previous model and the model on the current training domain.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 32 of its 72 references that are in the map. First posted on arXiv 8 Apr 2023.

Builds on

32 of 72 references.

References · 32

  1. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  2. 2016Progressive Neural Networkscited by 1,361
  3. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,330
  4. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,105
  5. 2021Characterizing possible failure modes in physics-informed neural networkscited by 643
  6. 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 634
  7. 2022Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's nextcited by 608
  8. 2021Physics-informed neural networks (PINNs) for fluid mechanics: A reviewcited by 473
  9. 2020AdapterHub: A Framework for Adapting Transformerscited by 444
  10. 2020BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learningcited by 371
  11. 2017Meta Networkscited by 269
  12. 2022A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networkscited by 236
20 more
  1. 2020Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanismcited by 233
  2. 2018Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselinescited by 228
  3. 2020Solving Allen-Cahn and Cahn-Hilliard Equations using the Adaptive Physics Informed Neural Networkscited by 123
  4. 2019PPINN: Parareal Physics-Informed Neural Network for time-dependent PDEscited by 118
  5. 2021Efficient training of physics-informed neural networks via importance samplingcited by 110
  6. 2022Multifidelity deep neural operators for efficient learning of partial differential equations with application to fast inverse design of nanoscale heat transportcited by 92
  7. 2021Long-time integration of parametric evolution equations with physics-informed DeepONetscited by 89
  8. 2019Physics-Informed Neural Networks for Power Systemscited by 80
  9. 2021Rehearsal revealed: The limits and merits of revisiting samples in continual learningcited by 76
  10. 2022Failure-informed adaptive sampling for PINNscited by 64
  11. 2022Multifidelity Deep Operator Networks For Data-Driven and Physics-Informed Problemscited by 58
  12. 2021DAE-PINN: A Physics-Informed Neural Network Model for Simulating Differential-Algebraic Equations with Application to Power Networkscited by 28
  13. 2021Galerkin Neural Networks: A Framework for Approximating Variational Equations with Error Controlcited by 24
  14. 2022On the Role of Fixed Points of Dynamical Systems in Training Physics-Informed Neural Networkscited by 23
  15. 2020Importance Driven Continual Learning for Segmentation Across Domainscited by 21
  16. 2021Recipes for when Physics Fails: Recovering Robust Learning of Physics Informed Neural Networkscited by 18
  17. 2021Applying Machine Learning to Study Fluid Mechanicscited by 13
  18. 2023A Multifidelity deep operator network approach to closure for multiscale systemscited by 11
  19. 2022Continual Learning of Dynamical Systems with Competitive Federated Reservoir Computingcited by 6
  20. 2023Continual HyperTransformer: A Meta-Learner for Continual Few-Shot Learningcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 5; 2025: 4; 2026: 3.

13 papers.

Sort:

  1. 2024Finite basis Kolmogorov-Arnold networks: domain decomposition for data-driven and physics-informed problemscited by 35
  2. 2023Stacked networks improve physics-informed training: applications to neural networks and deep operator networkscited by 21
  3. 2024Self-adaptive weights based on balanced residual decay rate for physics-informed neural networks and deep operator networkscited by 20
  4. 2024SPIKANs: Separable Physics-Informed Kolmogorov-Arnold Networkscited by 12
  5. 2024Multifidelity Kolmogorov-Arnold Networkscited by 9
  6. 2024Transfer Learning on Multi-Dimensional Data: A Novel Approach to Neural Network-Based Surrogate Modelingcited by 3
  7. 2025iPINNER: An Iterative Physics-Informed Neural Network with Ensemble Kalman Filtercited by 3
  8. 2025Physics-Informed DeepONet Coupled with FEM for Convective Transport in Porous Media with Sharp Gaussian Sourcescited by 3
  9. 2026SLE-FNO: Single-Layer Extensions for Task-Agnostic Continual Learning in Fourier Neural Operatorscited by 3
  10. 2025Bridging quantum and classical computing for partial differential equations through multifidelity machine learningcited by 1
  11. 2025Domain-Decomposed Graph Neural Network Surrogate Modeling for Ice Sheetscited by 0
  12. 2026Multifidelity-Augmented Gaussian Process Inputs for Surrogate Modeling from Scarce Datacited by 0
1 more
  1. 2026From Classification to Regression: Using a Fruitfly to Solve Equationscited by 0