# Multi-fidelity regression using artificial neural networks: efficient approximation of parameter-dependent output quantities

arXiv: https://arxiv.org/abs/2102.13403 · HTML: https://mlsanity.com/paper/2102.13403/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 47 references that are in the map.
- First posted on arXiv 26 Feb 2021.

## Builds on (most cited first, 11 of 11)

- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [Deep Neural Networks as Gaussian Processes](https://mlsanity.com/paper/1711.00165/) (2017)
- [Generalizing from a Few Examples: A Survey on Few-Shot Learning](https://mlsanity.com/paper/1904.05046/) (2019)
- [Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations](https://mlsanity.com/paper/1706.04702/) (2017)
- [Survey of multifidelity methods in uncertainty propagation, inference, and optimization](https://mlsanity.com/paper/1806.10761/) (2018)
- [A comprehensive deep learning-based approach to reduced order modeling of nonlinear time-dependent parametrized PDEs](https://mlsanity.com/paper/2001.04001/) (2020)
- [Multi-fidelity Bayesian Neural Networks: Algorithms and Applications](https://mlsanity.com/paper/2012.13294/) (2020)
- [Deep Gaussian Processes for Multi-fidelity Modeling](https://mlsanity.com/paper/1903.07320/) (2019)
- [Inferring solutions of differential equations using noisy multi-fidelity data](https://mlsanity.com/paper/1607.04805/) (2016)
- [Deep Multi-fidelity Gaussian Processes](https://mlsanity.com/paper/1604.07484/) (2016)
- [Preserving Lagrangian structure in nonlinear model reduction with application to structural dynamics](https://mlsanity.com/paper/1401.8044/) (2014)

## Built on by (most cited first, 30 of 31)

- [Multi-fidelity surrogate modeling using long short-term memory networks](https://mlsanity.com/paper/2208.03115/) (2022)
- [Multi-fidelity reduced-order surrogate modeling](https://mlsanity.com/paper/2309.00325/) (2023)
- [An extended physics informed neural network for preliminary analysis of parametric optimal control problems](https://mlsanity.com/paper/2110.13530/) (2021)
- [Multi-fidelity Hierarchical Neural Processes](https://mlsanity.com/paper/2206.04872/) (2022)
- [A DeepONet multi-fidelity approach for residual learning in reduced order modeling](https://mlsanity.com/paper/2302.12682/) (2023)
- [Multi-Fidelity Residual Neural Processes for Scalable Surrogate Modeling](https://mlsanity.com/paper/2402.18846/) (2024)
- [Disentangled Multi-Fidelity Deep Bayesian Active Learning](https://mlsanity.com/paper/2305.04392/) (2023)
- [GFN: A graph feedforward network for resolution-invariant reduced operator learning in multifidelity applications](https://mlsanity.com/paper/2406.03569/) (2024)
- [Multifidelity Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2410.14764/) (2024)
- [Feature-adjacent multi-fidelity physics-informed machine learning for partial differential equations](https://mlsanity.com/paper/2303.11577/) (2023)
- [Multifidelity data fusion in convolutional encoder/decoder networks](https://mlsanity.com/paper/2205.05187/) (2022)
- [Residual Multi-Fidelity Neural Network Computing](https://mlsanity.com/paper/2310.03572/) (2023)
- [Practical multi-fidelity machine learning: fusion of deterministic and Bayesian models](https://mlsanity.com/paper/2407.15110/) (2024)
- [Multifidelity linear regression for scientific machine learning from scarce data](https://mlsanity.com/paper/2403.08627/) (2024)
- [Multi-Fidelity Bayesian Neural Network for Uncertainty Quantification in Transonic Aerodynamic Loads](https://mlsanity.com/paper/2407.05684/) (2024)
- [Progressive multi-fidelity learning with neural networks for physical system predictions](https://mlsanity.com/paper/2510.13762/) (2025)
- [Surrogates for Physics-based and Data-driven Modelling of Parametric Systems: Review and New Perspectives](https://mlsanity.com/paper/2603.12870/) (2026)
- [On the performance of multi-fidelity and reduced-dimensional neural emulators for inference of physiological boundary conditions](https://mlsanity.com/paper/2506.11683/) (2025)
- [Bridging quantum and classical computing for partial differential equations through multifidelity machine learning](https://mlsanity.com/paper/2512.05241/) (2025)
- [Multi-Fidelity Delayed Acceptance: hierarchical MCMC sampling for Bayesian inverse problems combining multiple solvers through deep neural networks](https://mlsanity.com/paper/2512.16430/) (2025)
- [Multifidelity Surrogate Modeling of Depressurized Loss of Forced Cooling in High-temperature Gas Reactors](https://mlsanity.com/paper/2603.14143/) (2026)
- [Multi-Fidelity SINDy: Sparse Discovery of Nonlinear Dynamical Systems with Fidelity-Weighted Measurements](https://mlsanity.com/paper/2606.15690/) (2026)
- [An iterative multi-fidelity approach for model order reduction of multi-dimensional input parametric PDE systems](https://mlsanity.com/paper/2301.09483/) (2023)
- [Differentiable Multi-Fidelity Fusion: Efficient Learning of Physics Simulations with Neural Architecture Search and Transfer Learning](https://mlsanity.com/paper/2306.06904/) (2023)
- [Open-Source High-Speed Flight Surrogate Modeling Framework](https://mlsanity.com/paper/2411.03598/) (2024)
- [Projection-based multifidelity linear regression for data-scarce applications](https://mlsanity.com/paper/2508.08517/) (2025)
- [Personalized Treatment Outcome Prediction from Scarce Data via Dual-Channel Knowledge Distillation and Adaptive Fusion](https://mlsanity.com/paper/2510.26444/) (2025)
- [Multi-fidelity aerodynamic data fusion by autoencoder transfer learning](https://mlsanity.com/paper/2512.13069/) (2025)
- [FilDeep: Learning Large Deformations of Elastic-Plastic Solids with Multi-Fidelity Data](https://mlsanity.com/paper/2601.10031/) (2026)
- [Multi-Fidelity Physics-Informed Neural Networks with Bayesian Uncertainty Quantification and Adaptive Residual Learning for Efficient Solution of Parametric Partial Differential Equations](https://mlsanity.com/paper/2602.01176/) (2026)

## Abstract (arXiv)

Highly accurate numerical or physical experiments are often time-consuming or expensive to obtain. When time or budget restrictions prohibit the generation of additional data, the amount of available samples may be too limited to provide satisfactory model results. Multi-fidelity methods deal with such problems by incorporating information from other sources, which are ideally well-correlated with the high-fidelity data, but can be obtained at a lower cost. By leveraging correlations between different data sets, multi-fidelity methods often yield superior generalization when compared to models based solely on a small amount of high-fidelity data. In this work, we present the use of artificial neural networks applied to multi-fidelity regression problems. By elaborating a few existing approaches, we propose new neural network architectures for multi-fidelity regression. The introduced models are compared against a traditional multi-fidelity scheme, co-kriging. A collection of artificial benchmarks are presented to measure the performance of the analyzed models. The results show that cross-validation in combination with Bayesian optimization consistently leads to neural network models that outperform the co-kriging scheme. Additionally, we show an application of multi-fidelity regression to an engineering problem. The propagation of a pressure wave into an acoustic horn with parametrized shape and frequency is considered, and the index of reflection intensity is approximated using the multi-fidelity models. A finite element model and a reduced basis model are adopted as the high- and low-fidelity, respectively. It is shown that the multi-fidelity neural network returns outputs that achieve a comparable accuracy to those from the expensive, full-order model, using only very few full-order evaluations combined with a larger amount of inaccurate but cheap evaluations of a reduced order model.