# Progressive reduced order modeling: empowering data-driven modeling with selective knowledge transfer

arXiv: https://arxiv.org/abs/2310.03770 · HTML: https://mlsanity.com/paper/2310.03770/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 47 references that are in the map.
- First posted on arXiv 4 Oct 2023.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [A Neural Network Solves, Explains, and Generates University Math Problems by Program Synthesis and Few-Shot Learning at Human Level](https://mlsanity.com/paper/2112.15594/) (2021)
- [Deep transfer operator learning for partial differential equations under conditional shift](https://mlsanity.com/paper/2204.09810/) (2022)
- [Learning two-phase microstructure evolution using neural operators and autoencoder architectures](https://mlsanity.com/paper/2204.07230/) (2022)
- [Real-time high-resolution CO2 geological storage prediction using nested Fourier neural operators](https://mlsanity.com/paper/2210.17051/) (2022)
- [A framework for data-driven solution and parameter estimation of PDEs using conditional generative adversarial networks](https://mlsanity.com/paper/2105.13136/) (2021)
- [Non-intrusive reduced order modeling of natural convection in porous media using convolutional autoencoders: comparison with linear subspace techniques](https://mlsanity.com/paper/2107.11460/) (2021)
- [Reduced order modeling for flow and transport problems with Barlow Twins self-supervised learning](https://mlsanity.com/paper/2202.05460/) (2022)
- [MFNets: Data efficient all-at-once learning of multifidelity surrogates as directed networks of information sources](https://mlsanity.com/paper/2008.02672/) (2020)
- [Physics-informed machine learning with differentiable programming for heterogeneous underground reservoir pressure management](https://mlsanity.com/paper/2206.10718/) (2022)
- [A Locally Conservative Mixed Finite Element Framework for Coupled Hydro-Mechanical-Chemical Processes in Heterogeneous Porous Media](https://mlsanity.com/paper/2010.04994/) (2020)
- [Data Assimilation Predictive GAN (DA-PredGAN): applied to determine the spread of COVID-19](https://mlsanity.com/paper/2105.07729/) (2021)
- [Gated Information Bottleneck for Generalization in Sequential Environments](https://mlsanity.com/paper/2110.06057/) (2021)
- [Gate Recurrent Unit Network based on Hilbert-Schmidt Independence Criterion for State-of-Health Estimation](https://mlsanity.com/paper/2303.09497/) (2023)

## Abstract (arXiv)

Data-driven modeling can suffer from a constant demand for data, leading to reduced accuracy and impractical for engineering applications due to the high cost and scarcity of information. To address this challenge, we propose a progressive reduced order modeling framework that minimizes data cravings and enhances data-driven modeling's practicality. Our approach selectively transfers knowledge from previously trained models through gates, similar to how humans selectively use valuable knowledge while ignoring unuseful information. By filtering relevant information from previous models, we can create a surrogate model with minimal turnaround time and a smaller training set that can still achieve high accuracy. We have tested our framework in several cases, including transport in porous media, gravity-driven flow, and finite deformation in hyperelastic materials. Our results illustrate that retaining information from previous models and utilizing a valuable portion of that knowledge can significantly improve the accuracy of the current model. We have demonstrated the importance of progressive knowledge transfer and its impact on model accuracy with reduced training samples. For instance, our framework with four parent models outperforms the no-parent counterpart trained on data nine times larger. Our research unlocks data-driven modeling's potential for practical engineering applications by mitigating the data scarcity issue. Our proposed framework is a significant step toward more efficient and cost-effective data-driven modeling, fostering advancements across various fields.