# MelissaDL x Breed: Towards Data-Efficient On-line Supervised Training of Multi-parametric Surrogates with Active Learning

arXiv: https://arxiv.org/abs/2410.05860 · HTML: https://mlsanity.com/paper/2410.05860/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 46 references that are in the map.
- First posted on arXiv 8 Oct 2024.

## Builds on (most cited first, 30 of 31)

- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Learning Mesh-Based Simulation with Graph Networks](https://mlsanity.com/paper/2010.03409/) (2020)
- [A Survey of Deep Active Learning](https://mlsanity.com/paper/2009.00236/) (2020)
- [The frontier of simulation-based inference](https://mlsanity.com/paper/1911.01429/) (2019)
- [ClimaX: A foundation model for weather and climate](https://mlsanity.com/paper/2301.10343/) (2023)
- [Not All Samples Are Created Equal: Deep Learning with Importance Sampling](https://mlsanity.com/paper/1803.00942/) (2018)
- [Neural General Circulation Models for Weather and Climate](https://mlsanity.com/paper/2311.07222/) (2023)
- [A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks](https://mlsanity.com/paper/2207.10289/) (2022)
- [Neural Operators for Accelerating Scientific Simulations and Design](https://mlsanity.com/paper/2309.15325/) (2023)
- [Poseidon: Efficient Foundation Models for PDEs](https://mlsanity.com/paper/2405.19101/) (2024)
- [An Expert's Guide to Training Physics-informed Neural Networks](https://mlsanity.com/paper/2308.08468/) (2023)
- [Efficient training of physics-informed neural networks via importance sampling](https://mlsanity.com/paper/2104.12325/) (2021)
- [Mitigating Propagation Failures in Physics-informed Neural Networks using Retain-Resample-Release (R3) Sampling](https://mlsanity.com/paper/2207.02338/) (2022)
- [Benchmarking Autoregressive Conditional Diffusion Models for Turbulent Flow Simulation](https://mlsanity.com/paper/2309.01745/) (2023)
- [Simulation Intelligence: Towards a New Generation of Scientific Methods](https://mlsanity.com/paper/2112.03235/) (2021)
- [Revisiting Methods for Finding Influential Examples](https://mlsanity.com/paper/2111.04683/) (2021)
- [Deep Active Learning by Leveraging Training Dynamics](https://mlsanity.com/paper/2110.08611/) (2021)
- [PINNACLE: PINN Adaptive ColLocation and Experimental points selection](https://mlsanity.com/paper/2404.07662/) (2024)
- [Active learning for data streams: a survey](https://mlsanity.com/paper/2302.08893/) (2023)
- [AUTOMATA: Gradient Based Data Subset Selection for Compute-Efficient Hyper-parameter Tuning](https://mlsanity.com/paper/2203.08212/) (2022)
- [Active Learning for Neural PDE Solvers](https://mlsanity.com/paper/2408.01536/) (2024)
- [Etalumis: Bringing Probabilistic Programming to Scientific Simulators at Scale](https://mlsanity.com/paper/1907.03382/) (2019)
- [BERT on a Data Diet: Finding Important Examples by Gradient-Based Pruning](https://mlsanity.com/paper/2211.05610/) (2022)
- [TiDAL: Learning Training Dynamics for Active Learning](https://mlsanity.com/paper/2210.06788/) (2022)
- [Colmena: Scalable Machine-Learning-Based Steering of Ensemble Simulations for High Performance Computing](https://mlsanity.com/paper/2110.02827/) (2021)
- [Deep Active Learning for Computer Vision: Past and Future](https://mlsanity.com/paper/2211.14819/) (2022)
- [Training Deep Surrogate Models with Large Scale Online Learning](https://mlsanity.com/paper/2306.16133/) (2023)
- [Reference Neural Operators: Learning the Smooth Dependence of Solutions of PDEs on Geometric Deformations](https://mlsanity.com/paper/2405.17509/) (2024)
- [Provably Neural Active Learning Succeeds via Prioritizing Perplexing Samples](https://mlsanity.com/paper/2406.03944/) (2024)
- [High Throughput Training of Deep Surrogates from Large Ensemble Runs](https://mlsanity.com/paper/2309.16743/) (2023)

## Built on by (most cited first, 1 of 1)

- [Learning Where to Simulate: Generative Active Sampling for Online PDE Surrogate Training](https://mlsanity.com/paper/2606.09949/) (2026)

## Abstract (arXiv)

Artificial intelligence is transforming scientific computing with deep neural network surrogates that approximate solutions to partial differential equations (PDEs). Traditional off-line training methods face issues with storage and I/O efficiency, as the training dataset has to be computed with numerical solvers up-front. Our previous work, the Melissa framework, addresses these problems by enabling data to be created "on-the-fly" and streamed directly into the training process. In this paper we introduce a new active learning method to enhance data-efficiency for on-line surrogate training. The surrogate is direct and multi-parametric, i.e., it is trained to predict a given timestep directly with different initial and boundary conditions parameters. Our approach uses Adaptive Multiple Importance Sampling guided by training loss statistics, in order to focus NN training on the difficult areas of the parameter space. Preliminary results for 2D heat PDE demonstrate the potential of this method, called Breed, to improve the generalization capabilities of surrogates while reducing computational overhead.