# Transfer Learning on Multi-Dimensional Data: A Novel Approach to Neural Network-Based Surrogate Modeling

arXiv: https://arxiv.org/abs/2410.12241 · HTML: https://mlsanity.com/paper/2410.12241/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 35 references that are in the map.
- First posted on arXiv 16 Oct 2024.

## Builds on (most cited first, 16 of 16)

- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Deep Learning Scaling is Predictable, Empirically](https://mlsanity.com/paper/1712.00409/) (2017)
- [Recent Advances in Convolutional Neural Networks](https://mlsanity.com/paper/1512.07108/) (2015)
- [On the Expressive Power of Deep Neural Networks](https://mlsanity.com/paper/1606.05336/) (2016)
- [Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification](https://mlsanity.com/paper/1801.06879/) (2018)
- [Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification](https://mlsanity.com/paper/1802.00850/) (2018)
- [Deep convolutional encoder-decoder networks for uncertainty quantification of dynamic multiphase flow in heterogeneous media](https://mlsanity.com/paper/1807.00882/) (2018)
- [Convergence Rates for Learning Linear Operators from Noisy Data](https://mlsanity.com/paper/2108.12515/) (2021)
- [Deep autoregressive neural networks for high-dimensional inverse problems in groundwater contaminant source identification](https://mlsanity.com/paper/1812.09444/) (2018)
- [Multi-scale Deep Neural Networks for Solving High Dimensional PDEs](https://mlsanity.com/paper/1910.11710/) (2019)
- [Transfer Learning on Multi-Fidelity Data](https://mlsanity.com/paper/2105.00856/) (2021)
- [A multifidelity approach to continual learning for physical systems](https://mlsanity.com/paper/2304.03894/) (2023)
- [Optimal deep learning of holomorphic operators between Banach spaces](https://mlsanity.com/paper/2406.13928/) (2024)
- [Feature-adjacent multi-fidelity physics-informed machine learning for partial differential equations](https://mlsanity.com/paper/2303.11577/) (2023)

## Built on by (most cited first, 3 of 3)

- [Multifidelity Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2410.14764/) (2024)
- [Physics-Informed Laplace Neural Operator for Solving Partial Differential Equations](https://mlsanity.com/paper/2602.12706/) (2026)
- [Domain-Decomposed Graph Neural Network Surrogate Modeling for Ice Sheets](https://mlsanity.com/paper/2512.01888/) (2025)

## Abstract (arXiv)

The development of efficient surrogates for partial differential equations (PDEs) is a critical step towards scalable modeling of complex, multiscale systems-of-systems. Convolutional neural networks (CNNs) have gained popularity as the basis for such surrogate models due to their success in capturing high-dimensional input-output mappings and the negligible cost of a forward pass. However, the high cost of generating training data -- typically via classical numerical solvers -- raises the question of whether these models are worth pursuing over more straightforward alternatives with well-established theoretical foundations, such as Monte Carlo methods. To reduce the cost of data generation, we propose training a CNN surrogate model on a mixture of numerical solutions to both the d-dimensional problem and its (d-1)-dimensional approximation, taking advantage of the efficiency savings guaranteed by the curse of dimensionality. We demonstrate our approach on a multiphase flow test problem, using transfer learning to train a dense fully-convolutional encoder-decoder CNN on the two classes of data. Numerical results from a sample uncertainty quantification task demonstrate that our surrogate model outperforms Monte Carlo with several times the data generation budget.