mlsanityData to 14 Oct 2026

Minimax Optimal Kernel Operator Learning via Multilevel Training

arXiv 2209.14430 ↗First posted 28 Sep 2022cs.LG cs.NA econ.EMcited by 13

Not on the map. Why?

Abstract (arXiv)

Learning mappings between infinite-dimensional function spaces has achieved empirical success in many disciplines of machine learning, including generative modeling, functional data analysis, causal inference, and multi-agent reinforcement learning. In this paper, we study the statistical limit of learning a Hilbert-Schmidt operator between two infinite-dimensional Sobolev reproducing kernel Hilbert spaces. We establish the information-theoretic lower bound in terms of the Sobolev Hilbert-Schmidt norm and show that a regularization that learns the spectral components below the bias contour and ignores the ones that are above the variance contour can achieve the optimal learning rate. At the same time, the spectral components between the bias and variance contours give us flexibility in designing computationally feasible machine learning algorithms. Based on this observation, we develop a multilevel kernel operator learning algorithm that is optimal when learning linear operators between infinite-dimensional function spaces.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 31 of its 77 references that are in the map. First posted on arXiv 28 Sep 2022.

Builds on

31 of 77 references.

References · 31

  1. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,908
  2. 2017DGM: A deep learning algorithm for solving partial differential equationscited by 678
  3. 2017The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problemscited by 583
  4. 2017Solving high-dimensional partial differential equations using deep learningcited by 556
  5. 2021Neural Operator: Learning Maps Between Function Spacescited by 414
  6. 2016Kernel Mean Embedding of Distributions: A Review and Beyondcited by 364
  7. 2017PDE-Net: Learning PDEs from Datacited by 311
  8. 2020Model Reduction and Neural Networks for Parametric PDEscited by 258
  9. 2018PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Networkcited by 210
  10. 2021Solving and Learning Nonlinear PDEs with Gaussian Processescited by 129
  11. 2019Kernel Instrumental Variable Regressioncited by 118
  12. 2017Sobolev Norm Learning Rates for Regularized Least-Squares Algorithmcited by 114
19 more
  1. 2020A Measure-Theoretic Approach to Kernel Conditional Mean Embeddingscited by 86
  2. 2020Minimax Estimation of Conditional Moment Modelscited by 83
  3. 2021Convergence Rates for Learning Linear Operators from Noisy Datacited by 55
  4. 2022Optimal Rates for Regularized Conditional Mean Embedding Learningcited by 50
  5. 2022Deep Nonparametric Estimation of Operators between Infinite Dimensional Spacescited by 45
  6. 2021Machine Learning For Elliptic PDEs: Fast Rate Generalization Bound, Neural Scaling Law and Minimax Optimalitycited by 40
  7. 2020Kernel Methods for Causal Functions: Dose, Heterogeneous, and Incremental Response Curvescited by 36
  8. 2021Meta-Auto-Decoder for Solving Parametric Partial Differential Equationscited by 36
  9. 2018Solving for high dimensional committor functions using artificial neural networkscited by 34
  10. 2022Learning Green's functions associated with time-dependent partial differential equationscited by 26
  11. 2020Breaking the Curse of Many Agents: Provable Mean Embedding Q-Iteration for Mean-Field Reinforcement Learningcited by 21
  12. 2020Nonparametric approximation of conditional expectation operatorscited by 21
  13. 2019Meta-learning Pseudo-differential Operators with Deep Neural Networkscited by 17
  14. 2019Kernel Conditional Density Operatorscited by 14
  15. 2019A Multi-level procedure for enhancing accuracy of machine learning algorithmscited by 12
  16. 2022Sobolev Acceleration and Statistical Optimality for Learning Elliptic Equations via Gradient Descentcited by 11
  17. 2020Functional Linear Regression with Mixed Predictorscited by 8
  18. 2021Sobolev Norm Learning Rates for Conditional Mean Embeddingscited by 7
  19. 2021Bayesian inference of an uncertain generalized diffusion operatorcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 3; 2024: 6; 2025: 2; 2026: 1.

13 papers.

Sort:

  1. 2024Operator Learning: Algorithms and Analysiscited by 85
  2. 2023A Mathematical Guide to Operator Learningcited by 76
  3. 2022Learning linear operators: Infinite-dimensional regression as a well-behaved non-compact inverse problemcited by 28
  4. 2023Error Bounds for Learning with Vector-Valued Random Featurescited by 23
  5. 2024Operator Learning Using Random Features: A Tool for Scientific Computingcited by 21
  6. 2024Optimal Rates for Vector-Valued Spectral Regularization Learning Algorithmscited by 12
  7. 2025Towards Sharp Minimax Risk Bounds for Operator Learningcited by 5
  8. 2025Multi-Level Monte Carlo Training of Neural Operatorscited by 4
  9. 2023Domain Generalization by Functional Regressioncited by 3
  10. 2024Spectrally Transformed Kernel Regressioncited by 2
  11. 2024Base Models for Parabolic Partial Differential Equationscited by 2
  12. 2026Optimal Multiscale Learning of Linear Operatorscited by 1
1 more
  1. 2024Benign overfitting in Fixed Dimension via Physics-Informed Learning with Smooth Inductive Biascited by 0