# Multifidelity linear regression for scientific machine learning from scarce data

arXiv: https://arxiv.org/abs/2403.08627 · HTML: https://mlsanity.com/paper/2403.08627/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 76 references that are in the map.
- First posted on arXiv 13 Mar 2024.

## Builds on (most cited first, 29 of 29)

- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Model Reduction and Neural Networks for Parametric PDEs](https://mlsanity.com/paper/2005.03180/) (2020)
- [Lift & Learn: Physics-informed machine learning for large-scale nonlinear dynamical systems](https://mlsanity.com/paper/1912.08177/) (2019)
- [Multifidelity deep neural operators for efficient learning of partial differential equations with application to fast inverse design of nanoscale heat transport](https://mlsanity.com/paper/2204.06684/) (2022)
- [The Random Feature Model for Input-Output Maps between Banach Spaces](https://mlsanity.com/paper/2005.10224/) (2020)
- [Multi-Information Source Optimization](https://mlsanity.com/paper/1603.00389/) (2016)
- [Derivative-Informed Neural Operator: An Efficient Framework for High-Dimensional Parametric Derivative Learning](https://mlsanity.com/paper/2206.10745/) (2022)
- [Multi-fidelity regression using artificial neural networks: efficient approximation of parameter-dependent output quantities](https://mlsanity.com/paper/2102.13403/) (2021)
- [Bi-fidelity Modeling of Uncertain and Partially Unknown Systems using DeepONets](https://mlsanity.com/paper/2204.00997/) (2022)
- [On transfer learning of neural networks using bi-fidelity data for uncertainty propagation](https://mlsanity.com/paper/2002.04495/) (2020)
- [Overview of Gaussian process based multi-fidelity techniques with variable relationship between fidelities](https://mlsanity.com/paper/2006.16728/) (2020)
- [Reduced operator inference for nonlinear partial differential equations](https://mlsanity.com/paper/2102.00083/) (2021)
- [Sampling weights of deep neural networks](https://mlsanity.com/paper/2306.16830/) (2023)
- [Efficient PDE-Constrained optimization under high-dimensional uncertainty using derivative-informed neural operators](https://mlsanity.com/paper/2305.20053/) (2023)
- [Neural Network Training Using ℓ1-Regularization and Bi-fidelity Data](https://mlsanity.com/paper/2105.13011/) (2021)
- [Multilevel Monte Carlo Variational Inference](https://mlsanity.com/paper/1902.00468/) (2019)
- [A DeepONet multi-fidelity approach for residual learning in reduced order modeling](https://mlsanity.com/paper/2302.12682/) (2023)
- [A support vector regression-based multi-fidelity surrogate model](https://mlsanity.com/paper/1906.09439/) (2019)
- [Use of Multifidelity Training Data and Transfer Learning for Efficient Construction of Subsurface Flow Surrogate Models](https://mlsanity.com/paper/2204.11138/) (2022)
- [A Multifidelity deep operator network approach to closure for multiscale systems](https://mlsanity.com/paper/2303.08893/) (2023)
- [Contour location via entropy reduction leveraging multiple information sources](https://mlsanity.com/paper/1805.07489/) (2018)
- [MFNets: Data efficient all-at-once learning of multifidelity surrogates as directed networks of information sources](https://mlsanity.com/paper/2008.02672/) (2020)
- [A Data-driven Multi-fidelity Physics-informed Learning Framework for Smart Manufacturing: A Composites Processing Case Study](https://mlsanity.com/paper/2202.06139/) (2022)
- [Context-aware surrogate modeling for balancing approximation and sampling costs in multi-fidelity importance sampling and Bayesian inverse problems](https://mlsanity.com/paper/2010.11708/) (2020)
- [Bayesian Deep Learning with Multilevel Trace-class Neural Networks](https://mlsanity.com/paper/2203.12961/) (2022)
- [mfEGRA: Multifidelity Efficient Global Reliability Analysis through Active Learning for Failure Boundary Location](https://mlsanity.com/paper/1910.02497/) (2019)
- [Context-aware learning of hierarchies of low-fidelity models for multi-fidelity uncertainty quantification](https://mlsanity.com/paper/2211.10835/) (2022)
- [Multivariate extensions of the Multilevel Best Linear Unbiased Estimator for ensemble-variational data assimilation](https://mlsanity.com/paper/2306.07017/) (2023)

## Built on by (most cited first, 3 of 3)

- [FIRE: Multi-fidelity Regression with Distribution-conditioned In-context Learning using Tabular Foundation Models](https://mlsanity.com/paper/2601.22371/) (2026)
- [Multi-Fidelity SINDy: Sparse Discovery of Nonlinear Dynamical Systems with Fidelity-Weighted Measurements](https://mlsanity.com/paper/2606.15690/) (2026)
- [Multifidelity-Augmented Gaussian Process Inputs for Surrogate Modeling from Scarce Data](https://mlsanity.com/paper/2603.22050/) (2026)

## Abstract (arXiv)

Machine learning (ML) methods, which fit to data the parameters of a given parameterized model class, have garnered significant interest as potential methods for learning surrogate models for complex engineering systems for which traditional simulation is expensive. However, in many scientific and engineering settings, generating high-fidelity data on which to train ML models is expensive, and the available budget for generating training data is limited, so that high-fidelity training data are scarce. ML models trained on scarce data have high variance, resulting in poor expected generalization performance. We propose a new multifidelity training approach for scientific machine learning via linear regression that exploits the scientific context where data of varying fidelities and costs are available: for example, high-fidelity data may be generated by an expensive fully resolved physics simulation whereas lower-fidelity data may arise from a cheaper model based on simplifying assumptions. We use the multifidelity data within an approximate control variate framework to define new multifidelity Monte Carlo estimators for linear regression models. We provide bias and variance analysis of our new estimators that guarantee the approach's accuracy and improved robustness to scarce high-fidelity data. Numerical results demonstrate that our multifidelity training approach achieves similar accuracy to the standard high-fidelity only approach with orders-of-magnitude reduced high-fidelity data requirements.