# Performance of Neural and Polynomial Operator Surrogates

arXiv: https://arxiv.org/abs/2604.00689 · HTML: https://mlsanity.com/paper/2604.00689/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 92 references that are in the map.
- First posted on arXiv 1 Apr 2026.

## Builds on (most cited first, 30 of 34)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [Neural Operator: Graph Kernel Network for Partial Differential Equations](https://mlsanity.com/paper/2003.03485/) (2020)
- [Physics-Informed Neural Operator for Learning Partial Differential Equations](https://mlsanity.com/paper/2111.03794/) (2021)
- [Fourier Neural Operator with Learned Deformations for PDEs on General Geometries](https://mlsanity.com/paper/2207.05209/) (2022)
- [How Does Batch Normalization Help Optimization?](https://mlsanity.com/paper/1805.11604/) (2018)
- [Multipole Graph Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2006.09535/) (2020)
- [Model Reduction and Neural Networks for Parametric PDEs](https://mlsanity.com/paper/2005.03180/) (2020)
- [Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecast](https://mlsanity.com/paper/2211.02556/) (2022)
- [Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations](https://mlsanity.com/paper/2407.07218/) (2024)
- [Multi-Grid Tensorized Fourier Neural Operator for High-Resolution PDEs](https://mlsanity.com/paper/2310.00120/) (2023)
- [Derivative-Informed Neural Operator: An Efficient Framework for High-Dimensional Parametric Derivative Learning](https://mlsanity.com/paper/2206.10745/) (2022)
- [Operator learning with PCA-Net: upper and lower complexity bounds](https://mlsanity.com/paper/2303.16317/) (2023)
- [Nonlocality and Nonlinearity Implies Universality in Operator Learning](https://mlsanity.com/paper/2304.13221/) (2023)
- [Derivative-Informed Projected Neural Networks for High-Dimensional Parametric Maps Governed by PDEs](https://mlsanity.com/paper/2011.15110/) (2020)
- [Continuum Attention for Neural Operators](https://mlsanity.com/paper/2406.06486/) (2024)
- [Data Complexity Estimates for Operator Learning](https://mlsanity.com/paper/2405.15992/) (2024)
- [Deep Operator Network Approximation Rates for Lipschitz Operators](https://mlsanity.com/paper/2307.09835/) (2023)
- [An operator learning perspective on parameter-to-observable maps](https://mlsanity.com/paper/2402.06031/) (2024)
- [Discretization Error of Fourier Neural Operators](https://mlsanity.com/paper/2405.02221/) (2024)
- [Neural and spectral operator surrogates: unified construction and expression rate bounds](https://mlsanity.com/paper/2207.04950/) (2022)
- [Residual-based error correction for neural operator accelerated infinite-dimensional Bayesian inverse problems](https://mlsanity.com/paper/2210.03008/) (2022)
- [Efficient PDE-Constrained optimization under high-dimensional uncertainty using derivative-informed neural operators](https://mlsanity.com/paper/2305.20053/) (2023)
- [Learning High-Dimensional Parametric Maps via Reduced Basis Adaptive Residual Networks](https://mlsanity.com/paper/2112.07096/) (2021)
- [Demystifying Data-Driven Probabilistic Medium-Range Weather Forecasting](https://mlsanity.com/paper/2601.18111/) (2026)
- [Derivative-informed neural operator acceleration of geometric MCMC for infinite-dimensional Bayesian inverse problems](https://mlsanity.com/paper/2403.08220/) (2024)
- [LazyDINO: Fast, scalable, and efficiently amortized Bayesian inversion via structure-exploiting and surrogate-driven measure transport](https://mlsanity.com/paper/2411.12726/) (2024)

## Built on by (most cited first, 7 of 7)

- [Neural Shape Operator Surrogates -- Expression Rate Bounds](https://mlsanity.com/paper/2604.18012/) (2026)
- [Fast-Mixing Markov Chains without Gradients](https://mlsanity.com/paper/2606.27564/) (2026)
- [Enabling Real-Time Training of a Wildfire-to-Smoke Map with Multilinear Operators](https://mlsanity.com/paper/2605.04164/) (2026)
- [Universal Approximation of Nonlinear Operators and Their Derivatives](https://mlsanity.com/paper/2605.15285/) (2026)
- [Near-Optimal Learning of Gaussian Sobolev Operators](https://mlsanity.com/paper/2607.11921/) (2026)
- [No Free Lunch in Flow Surrogates under Time-Varying Boundary Conditions: A Two-Regime Study](https://mlsanity.com/paper/2607.23667/) (2026)
- [Into the danger zone: stable extrapolation in high-dimensional function and operator learning](https://mlsanity.com/paper/2609.36709/) (2026)

## Abstract (arXiv)

We consider the problem of constructing surrogate operators for parameter-to-solution maps arising from parametric partial differential equations, where repeated forward model evaluations are computationally expensive. We present a systematic empirical comparison of neural operator surrogates, including a reduced-basis neural operator trained with L2μ and H1μ objectives and the Fourier neural operator, against polynomial surrogate methods, specifically a reduced-basis sparse-grid surrogate and a reduced-basis tensor-train surrogate. All methods are evaluated on a linear parametric diffusion problem and a nonlinear parametric hyperelasticity problem, using input fields with algebraically decaying spectral coefficients at varying rates of decay s. To enable fair comparisons, we analyze ensembles of surrogate models generated by varying hyperparameters and compare the resulting Pareto frontiers of cost versus approximation accuracy, decomposing cost into contributions from data generation, setup, and evaluation. Our results show that no single method is universally superior. Polynomial surrogates achieve substantially better data efficiency for smooth input fields (s ≥ 2), with convergence rates for the sparse-grid surrogate in agreement with theoretical predictions. For rough inputs (s ≤ 1), the Fourier neural operator displays the fastest convergence rates. Derivative-informed training consistently improves data efficiency over standard L2μ training, providing a competitive alternative for rough inputs in the low-data regime when Jacobian information is available at reasonable cost. These findings highlight the importance of matching the surrogate methodology to the regularity of the problem as well as accuracy demands and computational constraints of the application.