# Towards Optimal Sobolev Norm Rates for the Vector-Valued Regularized Least-Squares Algorithm

arXiv: https://arxiv.org/abs/2312.07186 · HTML: https://mlsanity.com/paper/2312.07186/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 51 references that are in the map.
- First posted on arXiv 12 Dec 2023.

## Builds on (most cited first, 17 of 17)

- [Gaussian Processes and Kernel Methods: A Review on Connections and Equivalences](https://mlsanity.com/paper/1807.02582/) (2018)
- [Kernel Instrumental Variable Regression](https://mlsanity.com/paper/1906.00232/) (2019)
- [Sobolev Norm Learning Rates for Regularized Least-Squares Algorithm](https://mlsanity.com/paper/1702.07254/) (2017)
- [Less is More: Nystr"om Computational Regularization](https://mlsanity.com/paper/1507.04717/) (2015)
- [A Measure-Theoretic Approach to Kernel Conditional Mean Embeddings](https://mlsanity.com/paper/2002.03689/) (2020)
- [Optimal Rates For Regularization Of Statistical Inverse Learning Problems](https://mlsanity.com/paper/1604.04054/) (2016)
- [Learning Dynamical Systems via Koopman Operator Regression in Reproducing Kernel Hilbert Spaces](https://mlsanity.com/paper/2205.14027/) (2022)
- [Operator-valued Kernels for Learning from Functional Response Data](https://mlsanity.com/paper/1510.08231/) (2015)
- [Optimal Rates for Spectral Algorithms with Least-Squares Regression over Hilbert Spaces](https://mlsanity.com/paper/1801.06720/) (2018)
- [Proximal Causal Learning with Kernels: Two-Stage Estimation and Moment Restriction](https://mlsanity.com/paper/2105.04544/) (2021)
- [Optimal Rates for Regularized Conditional Mean Embedding Learning](https://mlsanity.com/paper/2208.01711/) (2022)
- [Kernel Methods for Causal Functions: Dose, Heterogeneous, and Incremental Response Curves](https://mlsanity.com/paper/2010.04855/) (2020)
- [A General Framework for Consistent Structured Prediction with Implicit Loss Embeddings](https://mlsanity.com/paper/2002.05424/) (2020)
- [A Consistent Regularization Approach for Structured Prediction](https://mlsanity.com/paper/1605.07588/) (2016)
- [Learning linear operators: Infinite-dimensional regression as a well-behaved non-compact inverse problem](https://mlsanity.com/paper/2211.08875/) (2022)
- [Nonparametric approximation of conditional expectation operators](https://mlsanity.com/paper/2012.12917/) (2020)
- [Nonlinear Meta-Learning Can Guarantee Faster Rates](https://mlsanity.com/paper/2307.10870/) (2023)

## Built on by (most cited first, 22 of 22)

- [Practical Kernel Tests of Conditional Independence](https://mlsanity.com/paper/2402.13196/) (2024)
- [Optimal Rates for Vector-Valued Spectral Regularization Learning Algorithms](https://mlsanity.com/paper/2405.14778/) (2024)
- [Nonparametric Instrumental Regression via Kernel Methods is Minimax Optimal](https://mlsanity.com/paper/2411.19653/) (2024)
- [On the Hardness of Conditional Independence Testing In Practice](https://mlsanity.com/paper/2512.14000/) (2025)
- [Sobolev Regularized MMD Gradient Flow](https://mlsanity.com/paper/2605.11884/) (2026)
- [Optimal Rates and Saturation for Noiseless Kernel Ridge Regression](https://mlsanity.com/paper/2402.15718/) (2024)
- [Optimality and Adaptivity of Deep Neural Features for Instrumental Variable Regression](https://mlsanity.com/paper/2501.04898/) (2025)
- [Regularized least squares learning with heavy-tailed noise is minimax optimal](https://mlsanity.com/paper/2505.14214/) (2025)
- [Doubly-Robust Estimation of Counterfactual Policy Mean Embeddings](https://mlsanity.com/paper/2506.02793/) (2025)
- [Interventional Processes for Causal Uncertainty Quantification](https://mlsanity.com/paper/2410.14483/) (2024)
- [Learning Theory for Kernel Bilevel Optimization](https://mlsanity.com/paper/2502.08457/) (2025)
- [The Generalised Kernel Covariance Measure](https://mlsanity.com/paper/2604.03721/) (2026)
- [Density Ratio-Free Doubly Robust Proxy Causal Learning](https://mlsanity.com/paper/2505.19807/) (2025)
- [kNNSampler: Stochastic Imputations for Recovering Missing Value Distributions](https://mlsanity.com/paper/2509.08366/) (2025)
- [A Kernel-based Stochastic Approximation Framework for Nonlinear Operator Learning](https://mlsanity.com/paper/2509.11070/) (2025)
- [Operator-Based Generalization Bound for Deep Learning: Insights on Multi-Task Learning](https://mlsanity.com/paper/2512.19184/) (2025)
- [On the Koopman-Based Generalization Bounds for Multi-Task Deep Learning](https://mlsanity.com/paper/2512.19199/) (2025)
- [Towards regularized learning from functional data with covariate shift](https://mlsanity.com/paper/2601.21019/) (2026)
- [Verifiable Regularity Criterion for Conditional Expectation Operators and Conditional Mean Embeddings with Applications to Nonparametric Regression, Bayesian Inverse Problems, and Koopman Operators](https://mlsanity.com/paper/2608.06155/) (2026)
- [Kernel conditional tests from learning-theoretic bounds](https://mlsanity.com/paper/2506.03898/) (2025)
- [Scalable Operator Learning via Nystr"om Approximation With Denoising Applications](https://mlsanity.com/paper/2606.26652/) (2026)
- [Learning-Theoretic Foundation for General Coded Computing: The Straggler Setting](https://mlsanity.com/paper/2608.28910/) (2026)

## Abstract (arXiv)

We present the first optimal rates for infinite-dimensional vector-valued ridge regression on a continuous scale of norms that interpolate between L2 and the hypothesis space, which we consider as a vector-valued reproducing kernel Hilbert space. These rates allow to treat the misspecified case in which the true regression function is not contained in the hypothesis space. We combine standard assumptions on the capacity of the hypothesis space with a novel tensor product construction of vector-valued interpolation spaces in order to characterize the smoothness of the regression function. Our upper bound not only attains the same rate as real-valued kernel ridge regression, but also removes the assumption that the target regression function is bounded. For the lower bound, we reduce the problem to the scalar setting using a projection argument. We show that these rates are optimal in most cases and independent of the dimension of the output space. We illustrate our results for the special case of vector-valued Sobolev spaces.