# Derivative-Informed Neural Operator: An Efficient Framework for High-Dimensional Parametric Derivative Learning

arXiv: https://arxiv.org/abs/2206.10745 · HTML: https://mlsanity.com/paper/2206.10745/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 73 references that are in the map.
- First posted on arXiv 21 Jun 2022.

## Builds on (most cited first, 30 of 32)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems](https://mlsanity.com/paper/1603.04467/) (2016)
- [Neural Operator: Learning Maps Between Function Spaces](https://mlsanity.com/paper/2108.08481/) (2021)
- [Multipole Graph Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2006.09535/) (2020)
- [Model Reduction and Neural Networks for Parametric PDEs](https://mlsanity.com/paper/2005.03180/) (2020)
- [Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems](https://mlsanity.com/paper/2111.02801/) (2021)
- [Sobolev Training for Neural Networks](https://mlsanity.com/paper/1706.04859/) (2017)
- [The Random Feature Model for Input-Output Maps between Banach Spaces](https://mlsanity.com/paper/2005.10224/) (2020)
- [POD-DL-ROM: enhancing deep learning-based reduced order models for nonlinear parametrized PDEs by proper orthogonal decomposition](https://mlsanity.com/paper/2101.11845/) (2021)
- [Derivative-Informed Projected Neural Networks for High-Dimensional Parametric Maps Governed by PDEs](https://mlsanity.com/paper/2011.15110/) (2020)
- [Dimension-independent likelihood-informed MCMC](https://mlsanity.com/paper/1411.3688/) (2014)
- [Projected Stein Variational Gradient Descent](https://mlsanity.com/paper/2002.03469/) (2020)
- [A computational framework for infinite-dimensional Bayesian inverse problems. Part I: The linearized case, with application to global seismic inversion](https://mlsanity.com/paper/1308.1313/) (2013)
- [Certified dimension reduction in nonlinear Bayesian inverse problems](https://mlsanity.com/paper/1807.03712/) (2018)
- [Projected Stein Variational Newton: A Fast and Scalable Bayesian Inference Method in High Dimensions](https://mlsanity.com/paper/1901.08659/) (2019)
- [A computational framework for infinite-dimensional Bayesian inverse problems: Part II. Stochastic Newton MCMC with application to ice sheet flow inverse problems](https://mlsanity.com/paper/1308.6221/) (2013)
- [Large-scale Bayesian optimal experimental design with derivative-informed projected neural network](https://mlsanity.com/paper/2201.07925/) (2022)
- [Learning High-Dimensional Parametric Maps via Reduced Basis Adaptive Residual Networks](https://mlsanity.com/paper/2112.07096/) (2021)
- [A Fast and Scalable Method for A-Optimal Design of Experiments for Infinite-dimensional Bayesian Nonlinear Inverse Problems](https://mlsanity.com/paper/1410.5899/) (2014)
- [Scalable and efficient algorithms for the propagation of uncertainty from data through inference to prediction for large-scale problems, with application to flow of the Antarctic ice sheet](https://mlsanity.com/paper/1410.1221/) (2014)
- [A fast and scalable computational framework for large-scale and high-dimensional Bayesian optimal experimental design](https://mlsanity.com/paper/2010.15196/) (2020)
- [A-optimal design of experiments for infinite-dimensional Bayesian linear inverse problems with regularized ℓ0-sparsification](https://mlsanity.com/paper/1308.4084/) (2013)
- [Optimal design of acoustic metamaterial cloaks under uncertainty](https://mlsanity.com/paper/2007.13252/) (2020)
- [Taylor approximation and variance reduction for PDE-constrained optimal control under uncertainty](https://mlsanity.com/paper/1804.04301/) (2018)
- [An efficient method for goal-oriented linear Bayesian optimal experimental design: Application to optimal sensor placemen](https://mlsanity.com/paper/2102.06627/) (2021)
- [Mean-variance risk-averse optimal control of systems governed by PDEs with random parameter fields using quadratic approximations](https://mlsanity.com/paper/1602.07592/) (2016)
- [hIPPYlib-MUQ: A Bayesian Inference Software Framework for Integration of Data with Complex Predictive Models under Uncertainty](https://mlsanity.com/paper/2112.00713/) (2021)

## Built on by (most cited first, 30 of 44)

- [Operator Learning: Algorithms and Analysis](https://mlsanity.com/paper/2402.15715/) (2024)
- [A Mathematical Guide to Operator Learning](https://mlsanity.com/paper/2312.14688/) (2023)
- [Deep Operator Network Approximation Rates for Lipschitz Operators](https://mlsanity.com/paper/2307.09835/) (2023)
- [Residual-based error correction for neural operator accelerated infinite-dimensional Bayesian inverse problems](https://mlsanity.com/paper/2210.03008/) (2022)
- [Efficient PDE-Constrained optimization under high-dimensional uncertainty using derivative-informed neural operators](https://mlsanity.com/paper/2305.20053/) (2023)
- [Derivative-enhanced Deep Operator Network](https://mlsanity.com/paper/2402.19242/) (2024)
- [Neural Dynamical Operator: Continuous Spatial-Temporal Model with Gradient-Based and Derivative-Free Optimization Methods](https://mlsanity.com/paper/2311.11798/) (2023)
- [CS4ML: A general framework for active learning with arbitrary data based on Christoffel functions](https://mlsanity.com/paper/2306.00945/) (2023)
- [Kernel Neural Operators (KNOs) for Scalable, Memory-efficient, Geometrically-flexible Operator Learning](https://mlsanity.com/paper/2407.00809/) (2024)
- [Enabling Local Neural Operators to perform Equation-Free System-Level Analysis](https://mlsanity.com/paper/2505.02308/) (2025)
- [Solving multiphysics-based inverse problems with learned surrogates and constraints](https://mlsanity.com/paper/2307.11099/) (2023)
- [BiLO: Bilevel Local Operator Learning for PDE Inverse Problems](https://mlsanity.com/paper/2404.17789/) (2024)
- [Variationally correct operator learning: Reduced basis neural operator with a posteriori error estimation](https://mlsanity.com/paper/2512.21319/) (2025)
- [Performance of Neural and Polynomial Operator Surrogates](https://mlsanity.com/paper/2604.00689/) (2026)
- [Optimal sampling for least-squares approximation](https://mlsanity.com/paper/2409.02342/) (2024)
- [PETScML: Second-order solvers for training regression problems in Scientific Machine Learning](https://mlsanity.com/paper/2403.12188/) (2024)
- [Neural Operator induced Gaussian Process framework for probabilistic solution of parametric partial differential equations](https://mlsanity.com/paper/2404.15618/) (2024)
- [Verification and Validation for Trustworthy Scientific Machine Learning](https://mlsanity.com/paper/2502.15496/) (2025)
- [Multifidelity linear regression for scientific machine learning from scarce data](https://mlsanity.com/paper/2403.08627/) (2024)
- [TAEN: A Model-Constrained Tikhonov Autoencoder Network for Forward and Inverse Problems](https://mlsanity.com/paper/2412.07010/) (2024)
- [Sobolev acceleration for neural networks](https://mlsanity.com/paper/2509.19773/) (2025)
- [F-Adapter: Frequency-Adaptive Parameter-Efficient Fine-Tuning in Scientific Machine Learning](https://mlsanity.com/paper/2509.23173/) (2025)
- [Method of Manufactured Learning for Solver-free Training of Neural Operators](https://mlsanity.com/paper/2511.12890/) (2025)
- [Geometric Neural Operators (GNPs) for Data-Driven Deep Learning of Non-Euclidean Operators](https://mlsanity.com/paper/2404.10843/) (2024)
- [Coupled Input-Output Dimension Reduction: Application to Goal-oriented Bayesian Experimental Design and Global Sensitivity Analysis](https://mlsanity.com/paper/2406.13425/) (2024)
- [Benchmarking Diffusion Annealing-Based Bayesian Inverse Problem Solvers](https://mlsanity.com/paper/2503.03007/) (2025)
- [Bayesian BiLO: Bilevel Local Operator Learning for Efficient Uncertainty Quantification of Bayesian PDE Inverse Problems with Low-Rank Adaptation](https://mlsanity.com/paper/2507.17019/) (2025)
- [Deep Hilbert--Galerkin Methods for Infinite-Dimensional PDEs and Optimal Control](https://mlsanity.com/paper/2603.19463/) (2026)
- [Accelerating PDE-Constrained Optimization by the Derivative of Neural Operators](https://mlsanity.com/paper/2506.13120/) (2025)
- [A reduced-order derivative-informed neural operator for subsurface fluid-flow](https://mlsanity.com/paper/2509.13620/) (2025)

## Abstract (arXiv)

We propose derivative-informed neural operators (DINOs), a general family of neural networks to approximate operators as infinite-dimensional mappings from input function spaces to output function spaces or quantities of interest. After discretizations both inputs and outputs are high-dimensional. We aim to approximate not only the operators with improved accuracy but also their derivatives (Jacobians) with respect to the input function-valued parameter to empower derivative-based algorithms in many applications, e.g., Bayesian inverse problems, optimization under parameter uncertainty, and optimal experimental design. The major difficulties include the computational cost of generating derivative training data and the high dimensionality of the problem leading to large training cost. To address these challenges, we exploit the intrinsic low-dimensionality of the derivatives and develop algorithms for compressing derivative information and efficiently imposing it in neural operator training yielding derivative-informed neural operators. We demonstrate that these advances can significantly reduce the costs of both data generation and training for large classes of problems (e.g., nonlinear steady state parametric PDE maps), making the costs marginal or comparable to the costs without using derivatives, and in particular independent of the discretization dimension of the input and output functions. Moreover, we show that the proposed DINO achieves significantly higher accuracy than neural operators trained without derivative information, for both function approximation and derivative approximation (e.g., Gauss-Newton Hessian), especially when the training data are limited.