# Multi-Fidelity Residual Neural Processes for Scalable Surrogate Modeling

arXiv: https://arxiv.org/abs/2402.18846 · HTML: https://mlsanity.com/paper/2402.18846/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 75 references that are in the map.
- First posted on arXiv 29 Feb 2024.

## Builds on (most cited first, 29 of 29)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Deep Kernel Learning](https://mlsanity.com/paper/1511.02222/) (2015)
- [Conditional Neural Processes](https://mlsanity.com/paper/1807.01613/) (2018)
- [Neural Processes](https://mlsanity.com/paper/1807.01622/) (2018)
- [Attentive Neural Processes](https://mlsanity.com/paper/1901.05761/) (2019)
- [Doubly Stochastic Variational Inference for Deep Gaussian Processes](https://mlsanity.com/paper/1705.08933/) (2017)
- [Survey of multifidelity methods in uncertainty propagation, inference, and optimization](https://mlsanity.com/paper/1806.10761/) (2018)
- [Learning to Control PDEs with Differentiable Physics](https://mlsanity.com/paper/2001.07457/) (2020)
- [Multi-fidelity Bayesian Optimisation with Continuous Approximations](https://mlsanity.com/paper/1703.06240/) (2017)
- [Multi-fidelity Bayesian Neural Networks: Algorithms and Applications](https://mlsanity.com/paper/2012.13294/) (2020)
- [Sequential Neural Processes](https://mlsanity.com/paper/1906.10264/) (2019)
- [A General Framework for Multi-fidelity Bayesian Optimization with Gaussian Processes](https://mlsanity.com/paper/1811.00755/) (2018)
- [Deep Gaussian Processes for Multi-fidelity Modeling](https://mlsanity.com/paper/1903.07320/) (2019)
- [Multi-Fidelity Bayesian Optimization via Deep Neural Networks](https://mlsanity.com/paper/2007.03117/) (2020)
- [The Functional Neural Process](https://mlsanity.com/paper/1906.08324/) (2019)
- [Doubly Stochastic Variational Inference for Neural Processes with Hierarchical Latent Variables](https://mlsanity.com/paper/2008.09469/) (2020)
- [Multi-fidelity regression using artificial neural networks: efficient approximation of parameter-dependent output quantities](https://mlsanity.com/paper/2102.13403/) (2021)
- [On transfer learning of neural networks using bi-fidelity data for uncertainty propagation](https://mlsanity.com/paper/2002.04495/) (2020)
- [Overview of Gaussian process based multi-fidelity techniques with variable relationship between fidelities](https://mlsanity.com/paper/2006.16728/) (2020)
- [The Neural Process Family: Survey, Applications and Perspectives](https://mlsanity.com/paper/2209.00517/) (2022)
- [Deep Multi-Fidelity Active Learning of High-dimensional Outputs](https://mlsanity.com/paper/2012.00901/) (2020)
- [Batch Multi-Fidelity Active Learning with Budget Constraints](https://mlsanity.com/paper/2210.12704/) (2022)
- [Multi-fidelity modeling with different input domain definitions using Deep Gaussian Processes](https://mlsanity.com/paper/2006.15924/) (2020)
- [Deep Multi-fidelity Gaussian Processes](https://mlsanity.com/paper/1604.07484/) (2016)
- [Multi-fidelity Hierarchical Neural Processes](https://mlsanity.com/paper/2206.04872/) (2022)
- [Disentangled Multi-Fidelity Deep Bayesian Active Learning](https://mlsanity.com/paper/2305.04392/) (2023)
- [Infinite-Fidelity Coregionalization for Physical Simulation](https://mlsanity.com/paper/2207.00678/) (2022)
- [Diffusion-Generative Multi-Fidelity Learning for Physical Simulation](https://mlsanity.com/paper/2311.05606/) (2023)
- [Multi-model Ensemble Analysis with Neural Network Gaussian Processes](https://mlsanity.com/paper/2202.04152/) (2022)

## Built on by (most cited first, 13 of 13)

- [Adapting While Learning: Grounding LLMs for Scientific Problems with Intelligent Tool Usage Adaptation](https://mlsanity.com/paper/2411.00412/) (2024)
- [Spatiotemporal Pyramid Flow Matching for Climate Emulation](https://mlsanity.com/paper/2512.02268/) (2025)
- [SimulRAG: Simulator-based RAG for Grounding LLMs in Long-form Scientific QA](https://mlsanity.com/paper/2509.25459/) (2025)
- [A Kernel-based Resource-efficient Neural Surrogate for Multi-fidelity Prediction of Aerodynamic Field](https://mlsanity.com/paper/2512.10287/) (2025)
- [Cheap2Rich: A Multi-Fidelity Framework for Data Assimilation and System Identification of Multiscale Physics -- Rotating Detonation Engines](https://mlsanity.com/paper/2601.20295/) (2026)
- [SMRS: advocating a unified reporting standard for surrogate models in the artificial intelligence era](https://mlsanity.com/paper/2502.06753/) (2025)
- [FIRE: Multi-fidelity Regression with Distribution-conditioned In-context Learning using Tabular Foundation Models](https://mlsanity.com/paper/2601.22371/) (2026)
- [MF-LAL: Drug Compound Generation Using Multi-Fidelity Latent Space Active Learning](https://mlsanity.com/paper/2410.11226/) (2024)
- [Multi-Fidelity SINDy: Sparse Discovery of Nonlinear Dynamical Systems with Fidelity-Weighted Measurements](https://mlsanity.com/paper/2606.15690/) (2026)
- [Real-Time Performance Analysis of Multi-Fidelity Residual Physics-Informed Neural Process-Based State Estimation for Robotic Systems](https://mlsanity.com/paper/2511.08231/) (2025)
- [Neural surrogates for designing gravitational wave detectors](https://mlsanity.com/paper/2511.19364/) (2025)
- [Multi-Fidelity Physics-Informed Neural Networks with Bayesian Uncertainty Quantification and Adaptive Residual Learning for Efficient Solution of Parametric Partial Differential Equations](https://mlsanity.com/paper/2602.01176/) (2026)
- [Sparsely-Supervised Data Assimilation via Physics-Informed Schr"odinger Bridge](https://mlsanity.com/paper/2603.22319/) (2026)

## Abstract (arXiv)

Multi-fidelity surrogate modeling aims to learn an accurate surrogate at the highest fidelity level by combining data from multiple sources. Traditional methods relying on Gaussian processes can hardly scale to high-dimensional data. Deep learning approaches utilize neural network based encoders and decoders to improve scalability. These approaches share encoded representations across fidelities without including corresponding decoder parameters. This hinders inference performance, especially in out-of-distribution scenarios when the highest fidelity data has limited domain coverage. To address these limitations, we propose Multi-fidelity Residual Neural Processes (MFRNP), a novel multi-fidelity surrogate modeling framework. MFRNP explicitly models the residual between the aggregated output from lower fidelities and ground truth at the highest fidelity. The aggregation introduces decoders into the information sharing step and optimizes lower fidelity decoders to accurately capture both in-fidelity and cross-fidelity information. We show that MFRNP significantly outperforms state-of-the-art in learning partial differential equations and a real-world climate modeling task. Our code is published at: https://github.com/Rose-STL-Lab/MFRNP