# Iterative Surrogate Model Optimization (ISMO): An active learning algorithm for PDE constrained optimization with deep neural networks

arXiv: https://arxiv.org/abs/2008.05730 · HTML: https://mlsanity.com/paper/2008.05730/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Aug 2020.

## Built on by (most cited first, 27 of 27)

- [Convolutional Neural Operators for robust and accurate learning of PDEs](https://mlsanity.com/paper/2302.01178/) (2023)
- [Poseidon: Efficient Foundation Models for PDEs](https://mlsanity.com/paper/2405.19101/) (2024)
- [Physics-Informed Machine Learning: A Survey on Problems, Methods and Applications](https://mlsanity.com/paper/2211.08064/) (2022)
- [On the approximation of functions by tanh neural networks](https://mlsanity.com/paper/2104.08938/) (2021)
- [Generic bounds on the approximation error for physics-informed (and) operator learning](https://mlsanity.com/paper/2205.11393/) (2022)
- [Error estimates for physics informed neural networks approximating the Navier-Stokes equations](https://mlsanity.com/paper/2203.09346/) (2022)
- [Error analysis for physics informed neural networks (PINNs) approximating Kolmogorov PDEs](https://mlsanity.com/paper/2106.14473/) (2021)
- [Nonlinear Reconstruction for Operator Learning of PDEs with Discontinuities](https://mlsanity.com/paper/2210.01074/) (2022)
- [wPINNs: Weak Physics informed neural networks for approximating entropy solutions of hyperbolic conservation laws](https://mlsanity.com/paper/2207.08483/) (2022)
- [An operator learning perspective on parameter-to-observable maps](https://mlsanity.com/paper/2402.06031/) (2024)
- [Beyond Regular Grids: Fourier-Based Neural Operators on Arbitrary Domains](https://mlsanity.com/paper/2305.19663/) (2023)
- [Neural Network Approaches for Parameterized Optimal Control](https://mlsanity.com/paper/2402.10033/) (2024)
- [Deep Adaptive Arbitrary Polynomial Chaos Expansion: A Mini-data-driven Semi-supervised Method for Uncertainty Quantification](https://mlsanity.com/paper/2107.10428/) (2021)
- [Failure-averse Active Learning for Physics-constrained Systems](https://mlsanity.com/paper/2110.14443/) (2021)
- [LSA-PINN: Linear Boundary Connectivity Loss for Solving PDEs on Complex Geometry](https://mlsanity.com/paper/2302.01518/) (2023)
- [VI-DGP: A variational inference method with deep generative prior for solving high-dimensional inverse problems](https://mlsanity.com/paper/2302.11173/) (2023)
- [Accelerated primal-dual methods with enlarged step sizes and operator learning for nonsmooth optimal control problems](https://mlsanity.com/paper/2307.00296/) (2023)
- [Feasibility Study on Active Learning of Smart Surrogates for Scientific Simulations](https://mlsanity.com/paper/2407.07674/) (2024)
- [Learning where to learn: Training data distribution optimization for scientific machine learning](https://mlsanity.com/paper/2505.21626/) (2025)
- [Error analysis for deep neural network approximations of parametric hyperbolic conservation laws](https://mlsanity.com/paper/2207.07362/) (2022)
- [Beyond Accuracy: EcoL2 Metric for Sustainable Neural PDE Solvers](https://mlsanity.com/paper/2505.12556/) (2025)
- [Learning WENO for entropy stable schemes to solve conservation laws](https://mlsanity.com/paper/2403.14848/) (2024)
- [Invertible Koopman neural operator for data-driven modeling of partial differential equations](https://mlsanity.com/paper/2503.19717/) (2025)
- [The ADMM-PINNs Algorithmic Framework for Nonsmooth PDE-Constrained Optimization: A Deep Learning Approach](https://mlsanity.com/paper/2302.08309/) (2023)
- [A numerical approach for the fractional Laplacian via deep neural networks](https://mlsanity.com/paper/2308.16272/) (2023)
- [An optimal Petrov-Galerkin framework for operator networks](https://mlsanity.com/paper/2503.04024/) (2025)
- [Learning task-specific predictive models for scientific computing](https://mlsanity.com/paper/2506.03835/) (2025)

## Abstract (arXiv)

We present a novel active learning algorithm, termed as iterative surrogate model optimization (ISMO), for robust and efficient numerical approximation of PDE constrained optimization problems. This algorithm is based on deep neural networks and its key feature is the iterative selection of training data through a feedback loop between deep neural networks and any underlying standard optimization algorithm. Under suitable hypotheses, we show that the resulting optimizers converge exponentially fast (and with exponentially decaying variance), with respect to increasing number of training samples. Numerical examples for optimal control, parameter identification and shape optimization problems for PDEs are provided to validate the proposed theory and to illustrate that ISMO significantly outperforms a standard deep neural network based surrogate optimization algorithm.