# A Kernel-based Stochastic Approximation Framework for Nonlinear Operator Learning

arXiv: https://arxiv.org/abs/2509.11070 · HTML: https://mlsanity.com/paper/2509.11070/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 48 references that are in the map.
- First posted on arXiv 14 Sep 2025.

## Builds on (most cited first, 28 of 28)

- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Model Reduction and Neural Networks for Parametric PDEs](https://mlsanity.com/paper/2005.03180/) (2020)
- [Kernel Instrumental Variable Regression](https://mlsanity.com/paper/1906.00232/) (2019)
- [Sobolev Norm Learning Rates for Regularized Least-Squares Algorithm](https://mlsanity.com/paper/1702.07254/) (2017)
- [A Measure-Theoretic Approach to Kernel Conditional Mean Embeddings](https://mlsanity.com/paper/2002.03689/) (2020)
- [Operator Learning: Algorithms and Analysis](https://mlsanity.com/paper/2402.15715/) (2024)
- [Learning Dynamical Systems via Koopman Operator Regression in Reproducing Kernel Hilbert Spaces](https://mlsanity.com/paper/2205.14027/) (2022)
- [Operator-valued Kernels for Learning from Functional Response Data](https://mlsanity.com/paper/1510.08231/) (2015)
- [Proximal Causal Learning with Kernels: Two-Stage Estimation and Moment Restriction](https://mlsanity.com/paper/2105.04544/) (2021)
- [Optimally tackling covariate shift in RKHS-based nonparametric regression](https://mlsanity.com/paper/2205.02986/) (2022)
- [Kernel Methods are Competitive for Operator Learning](https://mlsanity.com/paper/2304.13202/) (2023)
- [Deep Nonparametric Estimation of Operators between Infinite Dimensional Spaces](https://mlsanity.com/paper/2201.00217/) (2022)
- [A General Framework for Consistent Structured Prediction with Implicit Loss Embeddings](https://mlsanity.com/paper/2002.05424/) (2020)
- [A Consistent Regularization Approach for Structured Prediction](https://mlsanity.com/paper/1605.07588/) (2016)
- [Learning linear operators: Infinite-dimensional regression as a well-behaved non-compact inverse problem](https://mlsanity.com/paper/2211.08875/) (2022)
- [Towards Optimal Sobolev Norm Rates for the Vector-Valued Regularized Least-Squares Algorithm](https://mlsanity.com/paper/2312.07186/) (2023)
- [Operator Learning: A Statistical Perspective](https://mlsanity.com/paper/2504.03503/) (2025)
- [Optimal Rates for Vector-Valued Spectral Regularization Learning Algorithms](https://mlsanity.com/paper/2405.14778/) (2024)
- [Sobolev Acceleration and Statistical Optimality for Learning Elliptic Equations via Gradient Descent](https://mlsanity.com/paper/2205.07331/) (2022)
- [Capacity dependent analysis for functional online learning algorithms](https://mlsanity.com/paper/2209.12198/) (2022)
- [Optimal Convergence Rates for Neural Operators](https://mlsanity.com/paper/2412.17518/) (2024)
- [Data-Efficient Kernel Methods for Learning Differential Equations and Their Solution Operators: Algorithms and Error Analysis](https://mlsanity.com/paper/2503.01036/) (2025)
- [Vector-Valued Least-Squares Regression under Output Regularity Assumptions](https://mlsanity.com/paper/2211.08958/) (2022)
- [Neural Operators for Nonlinear Functionals on RKHS](https://mlsanity.com/paper/2403.12187/) (2024)
- [Stochastic Gradient Descent for Operator Learning in Hilbert Spaces: Convergence Rates and Minimax Lower Bounds](https://mlsanity.com/paper/2402.04691/) (2024)
- [Structured Prediction in Online Learning](https://mlsanity.com/paper/2406.12366/) (2024)
- [Learning Operators by Regularized Stochastic Gradient Descent with Operator-valued Kernels](https://mlsanity.com/paper/2504.18184/) (2025)

## Built on by (most cited first, 1 of 1)

- [Variation Spaces for Encoder--Decoder Neural Operators: Approximation and Generalization](https://mlsanity.com/paper/2606.01244/) (2026)

## Abstract (arXiv)

We develop a stochastic approximation framework for learning nonlinear operators between infinite-dimensional spaces utilizing general Mercer operator-valued kernels. Our framework encompasses two key classes: (i) operator-valued kernels whose associated integral operators are compact and hence admit discrete spectral decompositions, and (ii) separable kernels of the form K(x,x')=k(x,x')T, where k is a scalar-valued kernel and T is a positive operator on the output space. This broad setting induces expressive vector-valued reproducing kernel Hilbert spaces (RKHSs) that generalize the classical K=kI paradigm, thereby enabling rich structural modeling with rigorous theoretical guarantees. To address target operators lying outside the RKHS, we introduce vector-valued interpolation spaces to precisely quantify misspecification error. Within this framework, we establish non-asymptotic convergence rates for prediction, estimation, and misspecification errors in the online and finite-horizon settings. Importantly, the framework also accommodates a range of operator learning settings, from Fredholm integral operators to encoder--decoder architectures. Numerical experiments on the two-dimensional Navier--Stokes equations illustrate the proposed approach.