# Feasibility Study on Active Learning of Smart Surrogates for Scientific Simulations

arXiv: https://arxiv.org/abs/2407.07674 · HTML: https://mlsanity.com/paper/2407.07674/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 58 references that are in the map.
- First posted on arXiv 10 Jul 2024.

## Builds on (most cited first, 30 of 34)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds](https://mlsanity.com/paper/1906.03671/) (2019)
- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning](https://mlsanity.com/paper/1906.08158/) (2019)
- [Learning Loss for Active Learning](https://mlsanity.com/paper/1905.03677/) (2019)
- [Towards Multi-spatiotemporal-scale Generalized PDE Modeling](https://mlsanity.com/paper/2209.15616/) (2022)
- [PhyGeoNet: Physics-Informed Geometry-Adaptive Convolutional Neural Networks for Solving Parameterized Steady-State PDEs on Irregular Domain](https://mlsanity.com/paper/2004.13145/) (2020)
- [Cost-Effective Active Learning for Deep Image Classification](https://mlsanity.com/paper/1701.03551/) (2017)
- [Suggestive Annotation: A Deep Active Learning Framework for Biomedical Image Segmentation](https://mlsanity.com/paper/1706.04737/) (2017)
- [VAMPnets: Deep learning of molecular kinetics](https://mlsanity.com/paper/1710.06012/) (2017)
- [Deep Active Learning: Unified and Principled Method for Query and Training](https://mlsanity.com/paper/1911.09162/) (2019)
- [Deep Active Learning over the Long Tail](https://mlsanity.com/paper/1711.00941/) (2017)
- [Towards Robust and Reproducible Active Learning Using Neural Networks](https://mlsanity.com/paper/2002.09564/) (2020)
- [Semi-Supervised Active Learning with Temporal Output Discrepancy](https://mlsanity.com/paper/2107.14153/) (2021)
- [Effective Evaluation of Deep Active Learning on Image Classification Tasks](https://mlsanity.com/paper/2106.15324/) (2021)
- [Building high accuracy emulators for scientific simulations with deep neural architecture search](https://mlsanity.com/paper/2001.08055/) (2020)
- [Deep Active Learning by Leveraging Training Dynamics](https://mlsanity.com/paper/2110.08611/) (2021)
- [Active learning of deep surrogates for PDEs: Application to metasurface design](https://mlsanity.com/paper/2008.12649/) (2020)
- [Iterative Surrogate Model Optimization (ISMO): An active learning algorithm for PDE constrained optimization with deep neural networks](https://mlsanity.com/paper/2008.05730/) (2020)
- [Deep Learning the Physics of Transport Phenomena](https://mlsanity.com/paper/1709.02432/) (2017)
- [CosmoGAN: creating high-fidelity weak lensing convergence maps using Generative Adversarial Networks](https://mlsanity.com/paper/1706.02390/) (2017)
- [DeepDriveMD: Deep-Learning Driven Adaptive Molecular Simulations for Protein Folding](https://mlsanity.com/paper/1909.07817/) (2019)
- [An unsupervised learning approach to solving heat equations on chip based on Auto Encoder and Image Gradient](https://mlsanity.com/paper/2007.09684/) (2020)
- [Learning Everywhere: A Taxonomy for the Integration of Machine Learning and Simulations](https://mlsanity.com/paper/1909.13340/) (2019)
- [A Neural Pre-Conditioning Active Learning Algorithm to Reduce Label Complexity](https://mlsanity.com/paper/2104.03525/) (2021)
- [Deep Bayesian Active Learning for Accelerating Stochastic Simulation](https://mlsanity.com/paper/2106.02770/) (2021)
- [Learning Everywhere: Pervasive Machine Learning for Effective High-Performance Computation](https://mlsanity.com/paper/1902.10810/) (2019)
- [Efficient Neural PDE-Solvers using Quantization Aware Training](https://mlsanity.com/paper/2308.07350/) (2023)

## Built on by (most cited first, 4 of 4)

- [Active Learning for Neural PDE Solvers](https://mlsanity.com/paper/2408.01536/) (2024)
- [Active Learning with Selective Time-Step Acquisition for PDEs](https://mlsanity.com/paper/2511.18107/) (2025)
- [Learning Where to Simulate: Generative Active Sampling for Online PDE Surrogate Training](https://mlsanity.com/paper/2606.09949/) (2026)
- [Data-Efficient Neural Operator Training via Physics-Based Active Learning](https://mlsanity.com/paper/2605.21348/) (2026)

## Abstract (arXiv)

High-performance scientific simulations, important for comprehension of complex systems, encounter computational challenges especially when exploring extensive parameter spaces. There has been an increasing interest in developing deep neural networks (DNNs) as surrogate models capable of accelerating the simulations. However, existing approaches for training these DNN surrogates rely on extensive simulation data which are heuristically selected and generated with expensive computation -- a challenge under-explored in the literature. In this paper, we investigate the potential of incorporating active learning into DNN surrogate training. This allows intelligent and objective selection of training simulations, reducing the need to generate extensive simulation data as well as the dependency of the performance of DNN surrogates on pre-defined training simulations. In the problem context of constructing DNN surrogates for diffusion equations with sources, we examine the efficacy of diversity- and uncertainty-based strategies for selecting training simulations, considering two different DNN architecture. The results set the groundwork for developing the high-performance computing infrastructure for Smart Surrogates that supports on-the-fly generation of simulation data steered by active learning strategies to potentially improve the efficiency of scientific simulations.