# Spectral Algorithms under Covariate Shift

arXiv: https://arxiv.org/abs/2504.12625 · HTML: https://mlsanity.com/paper/2504.12625/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 32 references that are in the map.
- First posted on arXiv 17 Apr 2025.

## Builds on (most cited first, 7 of 7)

- [Rethinking Importance Weighting for Deep Learning under Distribution Shift](https://mlsanity.com/paper/2006.04662/) (2020)
- [Optimally tackling covariate shift in RKHS-based nonparametric regression](https://mlsanity.com/paper/2205.02986/) (2022)
- [When is Importance Weighting Correction Needed for Covariate Shift Adaptation?](https://mlsanity.com/paper/2303.04020/) (2023)
- [Towards a Unified Analysis of Kernel-based Methods Under Covariate Shift](https://mlsanity.com/paper/2310.08237/) (2023)
- [Capacity dependent analysis for functional online learning algorithms](https://mlsanity.com/paper/2209.12198/) (2022)
- [Adaptive learning of density ratios in RKHS](https://mlsanity.com/paper/2307.16164/) (2023)
- [General regularization in covariate shift adaptation](https://mlsanity.com/paper/2307.11503/) (2023)

## Built on by (most cited first, 3 of 3)

- [Spectral Algorithms in Misspecified Regression: Convergence under Covariate Shift](https://mlsanity.com/paper/2509.05106/) (2025)
- [Towards regularized learning from functional data with covariate shift](https://mlsanity.com/paper/2601.21019/) (2026)
- [Learning under Target Shift: Optimal Density Ratio Estimation and Importance-Weighted Regression](https://mlsanity.com/paper/2609.15785/) (2026)

## Abstract (arXiv)

Spectral algorithms leverage spectral regularization techniques to analyze and process data, providing a flexible framework for addressing supervised learning problems. To deepen our understanding of their performance in real-world scenarios where the distributions of training and test data may differ, we conduct a rigorous investigation into the convergence behavior of spectral algorithms under covariate shift. In this setting, the marginal distributions of the input data differ between the training and test datasets, while the conditional distribution of the output given the input remains unchanged. Within a non-parametric regression framework over a reproducing kernel Hilbert space, we analyze the convergence rates of spectral algorithms under covariate shift and show that they achieve minimax optimality when the density ratios between the training and test distributions are uniformly bounded. However, when these density ratios are unbounded, the spectral algorithms may become suboptimal. To address this issue, we propose a novel weighted spectral algorithm with normalized weights that incorporates density ratio information into the learning process. Our theoretical analysis shows that this normalized weighted approach achieves optimal capacity-independent convergence rates, but the rates will suffer from the saturation phenomenon. Furthermore, by introducing a weight clipping technique, we demonstrate that the convergence rates of the weighted spectral algorithm with clipped weights can approach the optimal capacity-dependent convergence rates arbitrarily closely. This improvement resolves the suboptimality issue in unbounded density ratio scenarios and advances the state-of-the-art by refining existing theoretical results.