mlsanityData to 13 Oct 2026

Optimal Convergence Rates for Neural Operators

arXiv 2412.17518 ↗First posted 23 Dec 2024stat.ML cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

We introduce the neural tangent kernel (NTK) regime for two-layer neural operators and analyze their generalization properties. For early-stopped gradient descent (GD), we derive fast convergence rates that are known to be minimax optimal within the framework of non-parametric regression in reproducing kernel Hilbert spaces (RKHS). We provide bounds on the number of hidden neurons and the number of second-stage samples necessary for generalization. To justify our NTK regime, we additionally show that any operator approximable by a neural operator can also be approximated by an operator from the RKHS. A key application of neural operators is learning surrogate maps for the solution operators of partial differential equations (PDEs). We consider the standard Poisson equation to illustrate our theoretical findings with simulations.

Key facts. Cited by 8 papers in the map (data through 13 Oct 2026). Builds on 18 of its 70 references that are in the map. First posted on arXiv 23 Dec 2024.

Builds on

18 of 70 references.

References · 18

  1. 2013Auto-Encoding Variational Bayescited by 4,024
  2. 2020Neural Operator: Graph Kernel Network for Partial Differential Equationscited by 728
  3. 2020Deep Equals Shallow for ReLU Networks in Kernel Regimescited by 84
  4. 2023A Mathematical Guide to Operator Learningcited by 75
  5. 2020Deep Neural Tangent Kernel and Laplace Kernel Have the Same RKHScited by 74
  6. 2022Learning linear operators: Infinite-dimensional regression as a well-behaved non-compact inverse problemcited by 28
  7. 2024Data Complexity Estimates for Operator Learningcited by 25
  8. 2024Toward a Better Understanding of Fourier Neural Operators from a Spectral Perspectivecited by 24
  9. 2018Optimal Convergence for Distributed Learning with Stochastic Gradient Methods and Spectral Algorithmscited by 22
  10. 2022Neural Tangent Kernel: A Surveycited by 22
  11. 2024Optimal Rates for Vector-Valued Spectral Regularization Learning Algorithmscited by 12
  12. 2024Kernel Neural Operators (KNOs) for Scalable, Memory-efficient, Geometrically-flexible Operator Learningcited by 11
6 more
  1. 2023How many Neurons do we need? A refined Analysis for Shallow Networks trained with Gradient Descentcited by 10
  2. 2024Nonlinear functional regression by functional deep neural network with kernel embeddingcited by 7
  3. 2024Neural-Kernel Conditional Mean Embeddingscited by 7
  4. 2024An Overview on Machine Learning Methods for Partial Differential Equations: from Physics Informed Neural Networks to Deep Operator Learningcited by 6
  5. 2021Deep Learning in High Dimension: Neural Network Approximation of Analytic Functions in L2(ℝd,γd)cited by 3
  6. 2024Convergence analysis of wide shallow neural operators within the framework of Neural Tangent Kernelcited by 2

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2025: 4; 2026: 4.

8 papers.

  1. 2025Continuous Representation Methods, Theories, and Applications: An Overview and Perspectivescited by 4
  2. 2025Random feature approximation for general spectral methodscited by 2
  3. 2025Thompson Sampling in Function Spaces via Neural Operatorscited by 1
  4. 2025A Kernel-based Stochastic Approximation Framework for Nonlinear Operator Learningcited by 1
  5. 2026Unveiling the Mechanism of Continuous Representation Full-Waveform Inversion: A Wave Based Neural Tangent Kernel Frameworkcited by 1
  6. 2026Random Features for Operator-Valued Kernels: Bridging Kernel Methods and Neural Operatorscited by 0
  7. 2026Ghost in the Kernel: In-Context Learning with Efficient Transformers via Domain Generalizationcited by 0
  8. 2026A Smoothed Discrepancy Principle for Random Feature Methods and Neural Networkscited by 0