# On transfer learning of neural networks using bi-fidelity data for uncertainty propagation

arXiv: https://arxiv.org/abs/2002.04495 · HTML: https://mlsanity.com/paper/2002.04495/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Feb 2020.

## Built on by (most cited first, 26 of 26)

- [Bi-fidelity Modeling of Uncertain and Partially Unknown Systems using DeepONets](https://mlsanity.com/paper/2204.00997/) (2022)
- [Neural Network Training Using ℓ1-Regularization and Bi-fidelity Data](https://mlsanity.com/paper/2105.13011/) (2021)
- [Multi-fidelity Hierarchical Neural Processes](https://mlsanity.com/paper/2206.04872/) (2022)
- [Multi-Fidelity Residual Neural Processes for Scalable Surrogate Modeling](https://mlsanity.com/paper/2402.18846/) (2024)
- [Multi-fidelity wavelet neural operator with application to uncertainty quantification](https://mlsanity.com/paper/2208.05606/) (2022)
- [Use of Multifidelity Training Data and Transfer Learning for Efficient Construction of Subsurface Flow Surrogate Models](https://mlsanity.com/paper/2204.11138/) (2022)
- [Disentangled Multi-Fidelity Deep Bayesian Active Learning](https://mlsanity.com/paper/2305.04392/) (2023)
- [Interpreting and generalizing deep learning in physics-based problems with functional linear models](https://mlsanity.com/paper/2307.04569/) (2023)
- [Multifidelity Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2410.14764/) (2024)
- [Feature-adjacent multi-fidelity physics-informed machine learning for partial differential equations](https://mlsanity.com/paper/2303.11577/) (2023)
- [Bi-fidelity Variational Auto-encoder for Uncertainty Quantification](https://mlsanity.com/paper/2305.16530/) (2023)
- [PINN surrogate of Li-ion battery models for parameter inference. Part I: Implementation and multi-fidelity hierarchies for the single-particle model](https://mlsanity.com/paper/2312.17329/) (2023)
- [Multifidelity data fusion in convolutional encoder/decoder networks](https://mlsanity.com/paper/2205.05187/) (2022)
- [Probabilistic Neural Data Fusion for Learning from an Arbitrary Number of Multi-fidelity Data Sets](https://mlsanity.com/paper/2301.13271/) (2023)
- [Data-Efficient Deep Operator Network for Unsteady Flow: A Multi-Fidelity Approach with Physics-Guided Subsampling](https://mlsanity.com/paper/2503.17941/) (2025)
- [A Physics-Guided Bi-Fidelity Fourier-Featured Operator Learning Framework for Predicting Time Evolution of Drag and Lift Coefficients](https://mlsanity.com/paper/2311.03639/) (2023)
- [Context-aware learning of hierarchies of low-fidelity models for multi-fidelity uncertainty quantification](https://mlsanity.com/paper/2211.10835/) (2022)
- [Multifidelity linear regression for scientific machine learning from scarce data](https://mlsanity.com/paper/2403.08627/) (2024)
- [Surrogates for Physics-based and Data-driven Modelling of Parametric Systems: Review and New Perspectives](https://mlsanity.com/paper/2603.12870/) (2026)
- [Graph Laplacian-based Bayesian Multi-fidelity Modeling](https://mlsanity.com/paper/2409.08211/) (2024)
- [Bridging quantum and classical computing for partial differential equations through multifidelity machine learning](https://mlsanity.com/paper/2512.05241/) (2025)
- [Prediction of Ultrasonic Guided Wave Propagation in Solid-fluid and their Interface under Uncertainty using Machine Learning](https://mlsanity.com/paper/2105.02813/) (2021)
- [A few-shot graph Laplacian-based approach for improving the accuracy of low-fidelity data](https://mlsanity.com/paper/2304.04862/) (2023)
- [A Bi-fidelity DeepONet Approach for Modeling Uncertain and Degrading Hysteretic Systems](https://mlsanity.com/paper/2304.12609/) (2023)
- [Personalized Treatment Outcome Prediction from Scarce Data via Dual-Channel Knowledge Distillation and Adaptive Fusion](https://mlsanity.com/paper/2510.26444/) (2025)
- [Towards Multi-Fidelity Scaling Laws of Neural Surrogates in CFD](https://mlsanity.com/paper/2511.01830/) (2025)

## Abstract (arXiv)

Due to their high degree of expressiveness, neural networks have recently been used as surrogate models for mapping inputs of an engineering system to outputs of interest. Once trained, neural networks are computationally inexpensive to evaluate and remove the need for repeated evaluations of computationally expensive models in uncertainty quantification applications. However, given the highly parameterized construction of neural networks, especially deep neural networks, accurate training often requires large amounts of simulation data that may not be available in the case of computationally expensive systems. In this paper, to alleviate this issue for uncertainty propagation, we explore the application of transfer learning techniques using training data generated from both high- and low-fidelity models. We explore two strategies for coupling these two datasets during the training procedure, namely, the standard transfer learning and the bi-fidelity weighted learning. In the former approach, a neural network model mapping the inputs to the outputs of interest is trained based on the low-fidelity data. The high-fidelity data is then used to adapt the parameters of the upper layer(s) of the low-fidelity network, or train a simpler neural network to map the output of the low-fidelity network to that of the high-fidelity model. In the latter approach, the entire low-fidelity network parameters are updated using data generated via a Gaussian process model trained with a small high-fidelity dataset. The parameter updates are performed via a variant of stochastic gradient descent with learning rates given by the Gaussian process model. Using three numerical examples, we illustrate the utility of these bi-fidelity transfer learning methods where we focus on accuracy improvement achieved by transfer learning over standard training approaches.