# Residual Multi-Fidelity Neural Network Computing

arXiv: https://arxiv.org/abs/2310.03572 · HTML: https://mlsanity.com/paper/2310.03572/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 57 references that are in the map.
- First posted on arXiv 5 Oct 2023.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Deep Learning in Neural Networks: An Overview](https://mlsanity.com/paper/1404.7828/) (2014)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems](https://mlsanity.com/paper/1710.00211/) (2017)
- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [Optimal approximation of piecewise smooth functions using deep ReLU neural networks](https://mlsanity.com/paper/1709.05289/) (2017)
- [Error bounds for approximations with deep ReLU neural networks in Ws,p norms](https://mlsanity.com/paper/1902.07896/) (2019)
- [Deep Neural Network Approximation Theory](https://mlsanity.com/paper/1901.02220/) (2019)
- [Algorithms for Solving High Dimensional PDEs: From Nonlinear Monte Carlo to Machine Learning](https://mlsanity.com/paper/2008.13333/) (2020)
- [A Theoretical Analysis of Deep Neural Networks and Parametric PDEs](https://mlsanity.com/paper/1904.00377/) (2019)
- [Multi-fidelity Bayesian Neural Networks: Algorithms and Applications](https://mlsanity.com/paper/2012.13294/) (2020)
- [Multifidelity Deep Operator Networks For Data-Driven and Physics-Informed Problems](https://mlsanity.com/paper/2204.09157/) (2022)
- [A proof that artificial neural networks overcome the curse of dimensionality in the numerical approximation of Black-Scholes partial differential equations](https://mlsanity.com/paper/1809.02362/) (2018)
- [The gap between theory and practice in function approximation with deep neural networks](https://mlsanity.com/paper/2001.07523/) (2020)
- [Nonlinear Approximation and (Deep) ReLU Networks](https://mlsanity.com/paper/1905.02199/) (2019)
- [A proof that deep artificial neural networks overcome the curse of dimensionality in the numerical approximation of Kolmogorov partial differential equations with constant diffusion and nonlinear drift coefficients](https://mlsanity.com/paper/1809.07321/) (2018)
- [Multi-fidelity regression using artificial neural networks: efficient approximation of parameter-dependent output quantities](https://mlsanity.com/paper/2102.13403/) (2021)
- [Deep neural network approximation for high-dimensional elliptic PDEs with boundary conditions](https://mlsanity.com/paper/2007.05384/) (2020)
- [Multi-fidelity surrogate modeling using long short-term memory networks](https://mlsanity.com/paper/2208.03115/) (2022)
- [Multifidelity data fusion in convolutional encoder/decoder networks](https://mlsanity.com/paper/2205.05187/) (2022)
- [Approximation Power of Deep Neural Networks: an explanatory mathematical survey](https://mlsanity.com/paper/2207.09511/) (2022)

## Built on by (most cited first, 5 of 5)

- [Approximation Error and Complexity Bounds for ReLU Networks on Low-Regular Function Spaces](https://mlsanity.com/paper/2405.06727/) (2024)
- [Multi-Fidelity SINDy: Sparse Discovery of Nonlinear Dynamical Systems with Fidelity-Weighted Measurements](https://mlsanity.com/paper/2606.15690/) (2026)
- [FilDeep: Learning Large Deformations of Elastic-Plastic Solids with Multi-Fidelity Data](https://mlsanity.com/paper/2601.10031/) (2026)
- [Multi-Fidelity Physics-Informed Neural Networks with Bayesian Uncertainty Quantification and Adaptive Residual Learning for Efficient Solution of Parametric Partial Differential Equations](https://mlsanity.com/paper/2602.01176/) (2026)
- [Toward Reliable Railway-Bogie Response Prediction Using Multifidelity TDNN and Physics-Informed Residual Learning](https://mlsanity.com/paper/2609.12018/) (2026)

## Abstract (arXiv)

In this work, we consider the general problem of constructing a neural network surrogate model using multi-fidelity information. Motivated by error-complexity estimates for ReLU neural networks, we formulate the correlation between an inexpensive low-fidelity model and an expensive high-fidelity model as a possibly non-linear residual function. This function defines a mapping between 1) the shared input space of the models along with the low-fidelity model output, and 2) the discrepancy between the outputs of the two models. The computational framework proceeds by training two neural networks to work in concert. The first network learns the residual function on a small set of high- and low-fidelity data. Once trained, this network is used to generate additional synthetic high-fidelity data, which is used in the training of the second network. The trained second network then acts as our surrogate for the high-fidelity quantity of interest. We present four numerical examples to demonstrate the power of the proposed framework, showing that significant savings in computational cost may be achieved when the output predictions are desired to be accurate within small tolerances.