# Fast PDE-constrained optimization via self-supervised operator learning

arXiv: https://arxiv.org/abs/2110.13297 · HTML: https://mlsanity.com/paper/2110.13297/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 70 references that are in the map.
- First posted on arXiv 25 Oct 2021.

## Builds on (most cited first, 16 of 16)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [When and why PINNs fail to train: A neural tangent kernel perspective](https://mlsanity.com/paper/2007.14527/) (2020)
- [Learning the solution operator of parametric partial differential equations with physics-informed DeepOnets](https://mlsanity.com/paper/2103.10974/) (2021)
- [On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks](https://mlsanity.com/paper/2012.10047/) (2020)
- [Physics-informed neural networks with hard constraints for inverse design](https://mlsanity.com/paper/2102.04626/) (2021)
- [DeepReach: A Deep Learning Approach to High-Dimensional Reachability](https://mlsanity.com/paper/2011.02082/) (2020)
- [Long-time integration of parametric evolution equations with physics-informed DeepONets](https://mlsanity.com/paper/2106.05384/) (2021)
- [Improved architectures and training algorithms for deep operator networks](https://mlsanity.com/paper/2110.01654/) (2021)
- [Multi-level Convolutional Autoencoder Networks for Parametric Prediction of Spatio-temporal Dynamics](https://mlsanity.com/paper/1912.11114/) (2019)
- [Neural reparameterization improves structural optimization](https://mlsanity.com/paper/1909.04240/) (2019)
- [Deep Learning of Turbulent Scalar Mixing](https://mlsanity.com/paper/1811.07095/) (2018)
- [Learning Composable Energy Surrogates for PDE Order Reduction](https://mlsanity.com/paper/2005.06549/) (2020)
- [Automatic shape derivatives for transient PDEs in FEniCS and Firedrake](https://mlsanity.com/paper/2001.10058/) (2020)

## Built on by (most cited first, 25 of 25)

- [Physics-Informed Machine Learning: A Survey on Problems, Methods and Applications](https://mlsanity.com/paper/2211.08064/) (2022)
- [Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations](https://mlsanity.com/paper/2407.07218/) (2024)
- [Learning Operators with Coupled Attention](https://mlsanity.com/paper/2201.01032/) (2022)
- [Residual-based error correction for neural operator accelerated infinite-dimensional Bayesian inverse problems](https://mlsanity.com/paper/2210.03008/) (2022)
- [Physics-informed neural networks for PDE-constrained optimization and control](https://mlsanity.com/paper/2205.03377/) (2022)
- [Efficient PDE-Constrained optimization under high-dimensional uncertainty using derivative-informed neural operators](https://mlsanity.com/paper/2305.20053/) (2023)
- [Bi-level Physics-Informed Neural Networks for PDE Constrained Optimization using Broyden's Hypergradients](https://mlsanity.com/paper/2209.07075/) (2022)
- [A DeepONet multi-fidelity approach for residual learning in reduced order modeling](https://mlsanity.com/paper/2302.12682/) (2023)
- [Neural Network Approaches for Parameterized Optimal Control](https://mlsanity.com/paper/2402.10033/) (2024)
- [Deep Operator Networks for Bayesian Parameter Estimation in PDEs](https://mlsanity.com/paper/2501.10684/) (2025)
- [The Seismic Wavefield Common Task Framework](https://mlsanity.com/paper/2512.19927/) (2025)
- [Physics-guided Data Augmentation for Learning the Solution Operator of Linear Differential Equations](https://mlsanity.com/paper/2212.04100/) (2022)
- [Quantum Model-Discovery](https://mlsanity.com/paper/2111.06376/) (2021)
- [VI-DGP: A variational inference method with deep generative prior for solving high-dimensional inverse problems](https://mlsanity.com/paper/2302.11173/) (2023)
- [Accelerated primal-dual methods with enlarged step sizes and operator learning for nonsmooth optimal control problems](https://mlsanity.com/paper/2307.00296/) (2023)
- [Real-time optimal control of high-dimensional parametrized systems by deep learning-based reduced order models](https://mlsanity.com/paper/2409.05709/) (2024)
- [Verification and Validation for Trustworthy Scientific Machine Learning](https://mlsanity.com/paper/2502.15496/) (2025)
- [Neural Born Series Operator for Biomedical Ultrasound Computed Tomography](https://mlsanity.com/paper/2312.15575/) (2023)
- [Deep Neural ODE Operator Networks for PDEs](https://mlsanity.com/paper/2510.15651/) (2025)
- [Accelerating PDE-Constrained Optimization by the Derivative of Neural Operators](https://mlsanity.com/paper/2506.13120/) (2025)
- [Harnessing AI for Inverse Partial Differential Equation Problems: Past, Present, and Prospects](https://mlsanity.com/paper/2605.16966/) (2026)
- [The ADMM-PINNs Algorithmic Framework for Nonsmooth PDE-Constrained Optimization: A Deep Learning Approach](https://mlsanity.com/paper/2302.08309/) (2023)
- [Latent feedback control of distributed systems in multiple scenarios through deep learning-based reduced order models](https://mlsanity.com/paper/2412.09942/) (2024)
- [A Single-Loop Bilevel Deep Learning Method for Optimal Control of Obstacle Problems](https://mlsanity.com/paper/2601.04120/) (2026)
- [Real-time optimal control with shallow recurrent decoder networks](https://mlsanity.com/paper/2607.19302/) (2026)

## Abstract (arXiv)

Design and optimal control problems are among the fundamental, ubiquitous tasks we face in science and engineering. In both cases, we aim to represent and optimize an unknown (black-box) function that associates a performance/outcome to a set of controllable variables through an experiment. In cases where the experimental dynamics can be described by partial differential equations (PDEs), such problems can be mathematically translated into PDE-constrained optimization tasks, which quickly become intractable as the number of control variables and the cost of experiments increases. In this work we leverage physics-informed deep operator networks (DeepONets) -- a self-supervised framework for learning the solution operator of parametric PDEs -- to build fast and differentiable surrogates for rapidly solving PDE-constrained optimization problems, even in the absence of any paired input-output training data. The effectiveness of the proposed framework will be demonstrated across different applications involving continuous functions as control or design variables, including time-dependent optimal control of heat transfer, and drag minimization of obstacles in Stokes flow. In all cases, we observe that DeepONets can minimize high-dimensional cost functionals in a matter of seconds, yielding a significant speed up compared to traditional adjoint PDE solvers that are typically costly and limited to relatively low-dimensional control/design parametrizations.