mlsanityData to 7 Oct 2026

Optimal Rates for Vector-Valued Spectral Regularization Learning Algorithms

arXiv 2405.14778 ↗First posted 23 May 2024stat.ML cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

We study theoretical properties of a broad class of regularized algorithms with vector-valued output. These spectral algorithms include kernel ridge regression, kernel principal component regression, various implementations of gradient descent and many more. Our contributions are twofold. First, we rigorously confirm the so-called saturation effect for ridge regression with vector-valued output by deriving a novel lower bound on learning rates; this bound is shown to be suboptimal when the smoothness of the regression function exceeds a certain level. Second, we present the upper bound for the finite sample risk general vector-valued spectral algorithms, applicable to both well-specified and misspecified scenarios (where the true regression function lies outside of the hypothesis space) which is minimax optimal in various regimes. All of our results explicitly allow the case of infinite-dimensional output variables, proving consistency of recent practical applications.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 24 of its 58 references that are in the map. First posted on arXiv 23 May 2024.

Builds on

24 of 58 references.

References · 24

  1. 2019Kernel Instrumental Variable Regressioncited by 118
  2. 2017Sobolev Norm Learning Rates for Regularized Least-Squares Algorithmcited by 113
  3. 2020A Measure-Theoretic Approach to Kernel Conditional Mean Embeddingscited by 86
  4. 2016Optimal Rates For Regularization Of Statistical Inverse Learning Problemscited by 68
  5. 2022Learning Dynamical Systems via Koopman Operator Regression in Reproducing Kernel Hilbert Spacescited by 66
  6. 2015Operator-valued Kernels for Learning from Functional Response Datacited by 59
  7. 2018Optimal Rates for Spectral Algorithms with Least-Squares Regression over Hilbert Spacescited by 59
  8. 2021Proximal Causal Learning with Kernels: Two-Stage Estimation and Moment Restrictioncited by 54
  9. 2022Optimal Rates for Regularized Conditional Mean Embedding Learningcited by 50
  10. 2020Kernel Methods for Causal Functions: Dose, Heterogeneous, and Incremental Response Curvescited by 36
  11. 2020A General Framework for Consistent Structured Prediction with Implicit Loss Embeddingscited by 35
  12. 2023Sharp Spectral Rates for Koopman Operator Learningcited by 35
12 more
  1. 2016A Consistent Regularization Approach for Structured Predictioncited by 33
  2. 2023On the Optimality of Misspecified Spectral Algorithmscited by 28
  3. 2022Learning linear operators: Infinite-dimensional regression as a well-behaved non-compact inverse problemcited by 27
  4. 2023Error Bounds for Learning with Vector-Valued Random Featurescited by 23
  5. 2018Optimal Convergence for Distributed Learning with Stochastic Gradient Methods and Spectral Algorithmscited by 22
  6. 2023Towards Optimal Sobolev Norm Rates for the Vector-Valued Regularized Least-Squares Algorithmcited by 22
  7. 2019Beating SGD Saturation with Tail-Averaging and Minibatchingcited by 21
  8. 2020Nonparametric approximation of conditional expectation operatorscited by 21
  9. 2023On the Optimality of Misspecified Kernel Ridge Regressioncited by 20
  10. 2022Minimax Optimal Kernel Operator Learning via Multilevel Trainingcited by 13
  11. 2024On the Saturation Effect of Kernel Ridge Regressioncited by 12
  12. 2024Generalization Error Curves for Analytic Spectral Algorithms under Power-law Decaycited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 5; 2026: 5.

12 papers.

  1. 2024Nonparametric Instrumental Regression via Kernel Methods is Minimax Optimalcited by 7
  2. 2024Optimal Convergence Rates for Neural Operatorscited by 7
  3. 2025Koopman-Equivariant Gaussian Processescited by 6
  4. 2025Regularized least squares learning with heavy-tailed noise is minimax optimalcited by 3
  5. 2026Sobolev Regularized MMD Gradient Flowcited by 3
  6. 2025Fixed-Gaussian Spectral Algorithms: Minimax Optimal Rates for Misspecified Learning and Transfercited by 2
  7. 2025Learning Theory for Kernel Bilevel Optimizationcited by 2
  8. 2026The Generalised Kernel Covariance Measurecited by 2
  9. 2025A Kernel-based Stochastic Approximation Framework for Nonlinear Operator Learningcited by 1
  10. 2026Towards regularized learning from functional data with covariate shiftcited by 1
  11. 2026Scalable Operator Learning via Nystr"om Approximation With Denoising Applicationscited by 0
  12. 2026Learning to control switching nonlinear systems with Koopman operator regressioncited by 0