# Optimal Convergence Rates for Neural Operators

arXiv: https://arxiv.org/abs/2412.17518 · HTML: https://mlsanity.com/paper/2412.17518/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 70 references that are in the map.
- First posted on arXiv 23 Dec 2024.

## Builds on (most cited first, 18 of 18)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Neural Operator: Graph Kernel Network for Partial Differential Equations](https://mlsanity.com/paper/2003.03485/) (2020)
- [Deep Equals Shallow for ReLU Networks in Kernel Regimes](https://mlsanity.com/paper/2009.14397/) (2020)
- [A Mathematical Guide to Operator Learning](https://mlsanity.com/paper/2312.14688/) (2023)
- [Deep Neural Tangent Kernel and Laplace Kernel Have the Same RKHS](https://mlsanity.com/paper/2009.10683/) (2020)
- [Learning linear operators: Infinite-dimensional regression as a well-behaved non-compact inverse problem](https://mlsanity.com/paper/2211.08875/) (2022)
- [Data Complexity Estimates for Operator Learning](https://mlsanity.com/paper/2405.15992/) (2024)
- [Toward a Better Understanding of Fourier Neural Operators from a Spectral Perspective](https://mlsanity.com/paper/2404.07200/) (2024)
- [Neural Tangent Kernel: A Survey](https://mlsanity.com/paper/2208.13614/) (2022)
- [Optimal Convergence for Distributed Learning with Stochastic Gradient Methods and Spectral Algorithms](https://mlsanity.com/paper/1801.07226/) (2018)
- [Optimal Rates for Vector-Valued Spectral Regularization Learning Algorithms](https://mlsanity.com/paper/2405.14778/) (2024)
- [Kernel Neural Operators (KNOs) for Scalable, Memory-efficient, Geometrically-flexible Operator Learning](https://mlsanity.com/paper/2407.00809/) (2024)
- [How many Neurons do we need? A refined Analysis for Shallow Networks trained with Gradient Descent](https://mlsanity.com/paper/2309.08044/) (2023)
- [Nonlinear functional regression by functional deep neural network with kernel embedding](https://mlsanity.com/paper/2401.02890/) (2024)
- [Neural-Kernel Conditional Mean Embeddings](https://mlsanity.com/paper/2403.10859/) (2024)
- [An Overview on Machine Learning Methods for Partial Differential Equations: from Physics Informed Neural Networks to Deep Operator Learning](https://mlsanity.com/paper/2408.13222/) (2024)
- [Deep Learning in High Dimension: Neural Network Approximation of Analytic Functions in L2(ℝd,γd)](https://mlsanity.com/paper/2111.07080/) (2021)
- [Convergence analysis of wide shallow neural operators within the framework of Neural Tangent Kernel](https://mlsanity.com/paper/2412.05545/) (2024)

## Built on by (most cited first, 8 of 8)

- [Continuous Representation Methods, Theories, and Applications: An Overview and Perspectives](https://mlsanity.com/paper/2505.15222/) (2025)
- [Random feature approximation for general spectral methods](https://mlsanity.com/paper/2506.16283/) (2025)
- [Thompson Sampling in Function Spaces via Neural Operators](https://mlsanity.com/paper/2506.21894/) (2025)
- [A Kernel-based Stochastic Approximation Framework for Nonlinear Operator Learning](https://mlsanity.com/paper/2509.11070/) (2025)
- [Random Features for Operator-Valued Kernels: Bridging Kernel Methods and Neural Operators](https://mlsanity.com/paper/2603.00971/) (2026)
- [Unveiling the Mechanism of Continuous Representation Full-Waveform Inversion: A Wave Based Neural Tangent Kernel Framework](https://mlsanity.com/paper/2603.22362/) (2026)
- [Ghost in the Kernel: In-Context Learning with Efficient Transformers via Domain Generalization](https://mlsanity.com/paper/2607.00479/) (2026)
- [A Smoothed Discrepancy Principle for Random Feature Methods and Neural Networks](https://mlsanity.com/paper/2609.21017/) (2026)

## Abstract (arXiv)

We introduce the neural tangent kernel (NTK) regime for two-layer neural operators and analyze their generalization properties. For early-stopped gradient descent (GD), we derive fast convergence rates that are known to be minimax optimal within the framework of non-parametric regression in reproducing kernel Hilbert spaces (RKHS). We provide bounds on the number of hidden neurons and the number of second-stage samples necessary for generalization. To justify our NTK regime, we additionally show that any operator approximable by a neural operator can also be approximated by an operator from the RKHS. A key application of neural operators is learning surrogate maps for the solution operators of partial differential equations (PDEs). We consider the standard Poisson equation to illustrate our theoretical findings with simulations.