mlsanityData to 7 Oct 2026

Towards Optimal Sobolev Norm Rates for the Vector-Valued Regularized Least-Squares Algorithm

arXiv 2312.07186 ↗First posted 12 Dec 2023stat.ML cs.LGcited by 22

Not on the map. Why?

Abstract (arXiv)

We present the first optimal rates for infinite-dimensional vector-valued ridge regression on a continuous scale of norms that interpolate between L2 and the hypothesis space, which we consider as a vector-valued reproducing kernel Hilbert space. These rates allow to treat the misspecified case in which the true regression function is not contained in the hypothesis space. We combine standard assumptions on the capacity of the hypothesis space with a novel tensor product construction of vector-valued interpolation spaces in order to characterize the smoothness of the regression function. Our upper bound not only attains the same rate as real-valued kernel ridge regression, but also removes the assumption that the target regression function is bounded. For the lower bound, we reduce the problem to the scalar setting using a projection argument. We show that these rates are optimal in most cases and independent of the dimension of the output space. We illustrate our results for the special case of vector-valued Sobolev spaces.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 17 of its 51 references that are in the map. First posted on arXiv 12 Dec 2023.

Builds on

17 of 51 references.

References · 17

  1. 2018Gaussian Processes and Kernel Methods: A Review on Connections and Equivalencescited by 188
  2. 2019Kernel Instrumental Variable Regressioncited by 118
  3. 2017Sobolev Norm Learning Rates for Regularized Least-Squares Algorithmcited by 113
  4. 2015Less is More: Nystr"om Computational Regularizationcited by 89
  5. 2020A Measure-Theoretic Approach to Kernel Conditional Mean Embeddingscited by 86
  6. 2016Optimal Rates For Regularization Of Statistical Inverse Learning Problemscited by 68
  7. 2022Learning Dynamical Systems via Koopman Operator Regression in Reproducing Kernel Hilbert Spacescited by 66
  8. 2015Operator-valued Kernels for Learning from Functional Response Datacited by 59
  9. 2018Optimal Rates for Spectral Algorithms with Least-Squares Regression over Hilbert Spacescited by 59
  10. 2021Proximal Causal Learning with Kernels: Two-Stage Estimation and Moment Restrictioncited by 54
  11. 2022Optimal Rates for Regularized Conditional Mean Embedding Learningcited by 50
  12. 2020Kernel Methods for Causal Functions: Dose, Heterogeneous, and Incremental Response Curvescited by 36
5 more
  1. 2020A General Framework for Consistent Structured Prediction with Implicit Loss Embeddingscited by 35
  2. 2016A Consistent Regularization Approach for Structured Predictioncited by 33
  3. 2022Learning linear operators: Infinite-dimensional regression as a well-behaved non-compact inverse problemcited by 27
  4. 2020Nonparametric approximation of conditional expectation operatorscited by 21
  5. 2023Nonlinear Meta-Learning Can Guarantee Faster Ratescited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 11; 2026: 6.

22 papers.

Sort:

  1. 2024Practical Kernel Tests of Conditional Independencecited by 13
  2. 2024Optimal Rates for Vector-Valued Spectral Regularization Learning Algorithmscited by 12
  3. 2024Nonparametric Instrumental Regression via Kernel Methods is Minimax Optimalcited by 7
  4. 2025On the Hardness of Conditional Independence Testing In Practicecited by 6
  5. 2024Optimal Rates and Saturation for Noiseless Kernel Ridge Regressioncited by 3
  6. 2025Optimality and Adaptivity of Deep Neural Features for Instrumental Variable Regressioncited by 3
  7. 2025Regularized least squares learning with heavy-tailed noise is minimax optimalcited by 3
  8. 2025Doubly-Robust Estimation of Counterfactual Policy Mean Embeddingscited by 3
  9. 2026Sobolev Regularized MMD Gradient Flowcited by 3
  10. 2024Interventional Processes for Causal Uncertainty Quantificationcited by 2
  11. 2025Learning Theory for Kernel Bilevel Optimizationcited by 2
  12. 2026The Generalised Kernel Covariance Measurecited by 2
10 more
  1. 2025Density Ratio-Free Doubly Robust Proxy Causal Learningcited by 1
  2. 2025kNNSampler: Stochastic Imputations for Recovering Missing Value Distributionscited by 1
  3. 2025A Kernel-based Stochastic Approximation Framework for Nonlinear Operator Learningcited by 1
  4. 2025Operator-Based Generalization Bound for Deep Learning: Insights on Multi-Task Learningcited by 1
  5. 2025On the Koopman-Based Generalization Bounds for Multi-Task Deep Learningcited by 1
  6. 2026Towards regularized learning from functional data with covariate shiftcited by 1
  7. 2026Verifiable Regularity Criterion for Conditional Expectation Operators and Conditional Mean Embeddings with Applications to Nonparametric Regression, Bayesian Inverse Problems, and Koopman Operatorscited by 1
  8. 2025Kernel conditional tests from learning-theoretic boundscited by 0
  9. 2026Scalable Operator Learning via Nystr"om Approximation With Denoising Applicationscited by 0
  10. 2026Learning-Theoretic Foundation for General Coded Computing: The Straggler Settingcited by 0