# Differentiable Multi-Fidelity Fusion: Efficient Learning of Physics Simulations with Neural Architecture Search and Transfer Learning

arXiv: https://arxiv.org/abs/2306.06904 · HTML: https://mlsanity.com/paper/2306.06904/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 49 references that are in the map.
- First posted on arXiv 12 Jun 2023.

## Builds on (most cited first, 17 of 17)

- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization](https://mlsanity.com/paper/1603.06560/) (2016)
- [Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration](https://mlsanity.com/paper/1811.00250/) (2018)
- [Machine Learning of Linear Differential Equations using Gaussian Processes](https://mlsanity.com/paper/1701.02440/) (2017)
- [Multi-Information Source Optimization](https://mlsanity.com/paper/1603.00389/) (2016)
- [Transfer learning based multi-fidelity physics informed deep neural network](https://mlsanity.com/paper/2005.10614/) (2020)
- [Multi-fidelity Bayesian Neural Networks: Algorithms and Applications](https://mlsanity.com/paper/2012.13294/) (2020)
- [Deep convolutional encoder-decoder networks for uncertainty quantification of dynamic multiphase flow in heterogeneous media](https://mlsanity.com/paper/1807.00882/) (2018)
- [A General Framework for Multi-fidelity Bayesian Optimization with Gaussian Processes](https://mlsanity.com/paper/1811.00755/) (2018)
- [Deep Gaussian Processes for Multi-fidelity Modeling](https://mlsanity.com/paper/1903.07320/) (2019)
- [Multi-Fidelity Bayesian Optimization via Deep Neural Networks](https://mlsanity.com/paper/2007.03117/) (2020)
- [Face Detection with the Faster R-CNN](https://mlsanity.com/paper/1606.03473/) (2016)
- [Multi-fidelity regression using artificial neural networks: efficient approximation of parameter-dependent output quantities](https://mlsanity.com/paper/2102.13403/) (2021)
- [Deep Multi-Fidelity Active Learning of High-dimensional Outputs](https://mlsanity.com/paper/2012.00901/) (2020)
- [Transfer Learning on Multi-Fidelity Data](https://mlsanity.com/paper/2105.00856/) (2021)
- [Parallel Architecture and Hyperparameter Search via Successive Halving and Classification](https://mlsanity.com/paper/1805.10255/) (2018)
- [GAR: Generalized Autoregression for Multi-Fidelity Fusion](https://mlsanity.com/paper/2301.05729/) (2023)

## Abstract (arXiv)

With rapid progress in deep learning, neural networks have been widely used in scientific research and engineering applications as surrogate models. Despite the great success of neural networks in fitting complex systems, two major challenges still remain: i) the lack of generalization on different problems/datasets, and ii) the demand for large amounts of simulation data that are computationally expensive. To resolve these challenges, we propose the differentiable mf (DMF) model, which leverages neural architecture search (NAS) to automatically search the suitable model architecture for different problems, and transfer learning to transfer the learned knowledge from low-fidelity (fast but inaccurate) data to high-fidelity (slow but accurate) model. Novel and latest machine learning techniques such as hyperparameters search and alternate learning are used to improve the efficiency and robustness of DMF. As a result, DMF can efficiently learn the physics simulations with only a few high-fidelity training samples, and outperform the state-of-the-art methods with a significant margin (with up to 58% improvement in RMSE) based on a variety of synthetic and practical benchmark problems.