# Transfer Learning on Multi-Fidelity Data

arXiv: https://arxiv.org/abs/2105.00856 · HTML: https://mlsanity.com/paper/2105.00856/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 43 references that are in the map.
- First posted on arXiv 29 Apr 2021.

## Builds on (most cited first, 12 of 12)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Deep Visual-Semantic Alignments for Generating Image Descriptions](https://mlsanity.com/paper/1412.2306/) (2014)
- [DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition](https://mlsanity.com/paper/1310.1531/) (2013)
- [Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data](https://mlsanity.com/paper/1901.06314/) (2019)
- [Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification](https://mlsanity.com/paper/1802.00850/) (2018)
- [A deep-learning-based surrogate model for data assimilation in dynamic subsurface flow problems](https://mlsanity.com/paper/1908.05823/) (2019)
- [Deep convolutional encoder-decoder networks for uncertainty quantification of dynamic multiphase flow in heterogeneous media](https://mlsanity.com/paper/1807.00882/) (2018)
- [Face Detection with the Faster R-CNN](https://mlsanity.com/paper/1606.03473/) (2016)
- [Deep autoregressive neural networks for high-dimensional inverse problems in groundwater contaminant source identification](https://mlsanity.com/paper/1812.09444/) (2018)
- [Multi-fidelity Generative Deep Learning Turbulent Flows](https://mlsanity.com/paper/2006.04731/) (2020)
- [Markov Chain Monte Carlo with Neural Network Surrogates: Application to Contaminant Source Identification](https://mlsanity.com/paper/2003.02322/) (2020)

## Built on by (most cited first, 18 of 18)

- [Multifidelity Deep Operator Networks For Data-Driven and Physics-Informed Problems](https://mlsanity.com/paper/2204.09157/) (2022)
- [Multi-fidelity surrogate modeling using long short-term memory networks](https://mlsanity.com/paper/2208.03115/) (2022)
- [Multi-fidelity wavelet neural operator with application to uncertainty quantification](https://mlsanity.com/paper/2208.05606/) (2022)
- [Use of Multifidelity Training Data and Transfer Learning for Efficient Construction of Subsurface Flow Surrogate Models](https://mlsanity.com/paper/2204.11138/) (2022)
- [Multi-fidelity prediction of fluid flow and temperature field based on transfer learning using Fourier Neural Operator](https://mlsanity.com/paper/2304.06972/) (2023)
- [Multifidelity Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2410.14764/) (2024)
- [Feature-adjacent multi-fidelity physics-informed machine learning for partial differential equations](https://mlsanity.com/paper/2303.11577/) (2023)
- [Multi-fidelity surrogate modeling for temperature field prediction using deep convolution neural network](https://mlsanity.com/paper/2301.06674/) (2023)
- [Multi-fidelity Fourier Neural Operator for Fast Modeling of Large-Scale Geological Carbon Storage](https://mlsanity.com/paper/2308.09113/) (2023)
- [Residuals Are Not Enough: Limits of Physics-Informed Pre-Training for Scientific Foundation Models](https://mlsanity.com/paper/2503.19081/) (2025)
- [Deep Learning for Simultaneous Inference of Hydraulic and Transport Properties](https://mlsanity.com/paper/2110.12367/) (2021)
- [Transfer Learning on Multi-Dimensional Data: A Novel Approach to Neural Network-Based Surrogate Modeling](https://mlsanity.com/paper/2410.12241/) (2024)
- [Robust Topology Optimization Using Multi-Fidelity Variational Autoencoders](https://mlsanity.com/paper/2107.10661/) (2021)
- [Assessing the performance of correlation-based multi-fidelity neural emulators](https://mlsanity.com/paper/2512.02868/) (2025)
- [Curriculum Learning for Mesh-based simulations](https://mlsanity.com/paper/2509.13138/) (2025)
- [Differentiable Multi-Fidelity Fusion: Efficient Learning of Physics Simulations with Neural Architecture Search and Transfer Learning](https://mlsanity.com/paper/2306.06904/) (2023)
- [Towards Multi-Fidelity Scaling Laws of Neural Surrogates in CFD](https://mlsanity.com/paper/2511.01830/) (2025)
- [ONE CYLinder: A Benchmark for Graph-Based Surrogate Modeling of Unsteady Bluff-Body Flows](https://mlsanity.com/paper/2609.08947/) (2026)

## Abstract (arXiv)

Neural networks (NNs) are often used as surrogates or emulators of partial differential equations (PDEs) that describe the dynamics of complex systems. A virtually negligible computational cost of such surrogates renders them an attractive tool for ensemble-based computation, which requires a large number of repeated PDE solves. Since the latter are also needed to generate sufficient data for NN training, the usefulness of NN-based surrogates hinges on the balance between the training cost and the computational gain stemming from their deployment. We rely on multi-fidelity simulations to reduce the cost of data generation for subsequent training of a deep convolutional NN (CNN) using transfer learning. High- and low-fidelity images are generated by solving PDEs on fine and coarse meshes, respectively. We use theoretical results for multilevel Monte Carlo to guide our choice of the numbers of images of each kind. We demonstrate the performance of this multi-fidelity training strategy on the problem of estimation of the distribution of a quantity of interest, whose dynamics is governed by a system of nonlinear PDEs (parabolic PDEs of multi-phase flow in heterogeneous porous media) with uncertain/random parameters. Our numerical experiments demonstrate that a mixture of a comparatively large number of low-fidelity data and smaller numbers of high- and low-fidelity data provides an optimal balance of computational speed-up and prediction accuracy. The former is reported relative to both CNN training on high-fidelity images only and Monte Carlo solution of the PDEs. The latter is expressed in terms of both the Wasserstein distance and the Kullback-Leibler divergence.