# Spectral Algorithms in Misspecified Regression: Convergence under Covariate Shift

arXiv: https://arxiv.org/abs/2509.05106 · HTML: https://mlsanity.com/paper/2509.05106/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 39 references that are in the map.
- First posted on arXiv 5 Sep 2025.

## Builds on (most cited first, 17 of 17)

- [What is the Effect of Importance Weighting in Deep Learning?](https://mlsanity.com/paper/1812.03372/) (2018)
- [Generalization Properties of Learning with Random Features](https://mlsanity.com/paper/1602.04474/) (2016)
- [Sobolev Norm Learning Rates for Regularized Least-Squares Algorithm](https://mlsanity.com/paper/1702.07254/) (2017)
- [Early stopping and non-parametric regression: An optimal data-dependent stopping rule](https://mlsanity.com/paper/1306.3574/) (2013)
- [Optimal Rates For Regularization Of Statistical Inverse Learning Problems](https://mlsanity.com/paper/1604.04054/) (2016)
- [Optimal Rates for Spectral Algorithms with Least-Squares Regression over Hilbert Spaces](https://mlsanity.com/paper/1801.06720/) (2018)
- [Optimally tackling covariate shift in RKHS-based nonparametric regression](https://mlsanity.com/paper/2205.02986/) (2022)
- [Optimal Rates for Regularized Conditional Mean Embedding Learning](https://mlsanity.com/paper/2208.01711/) (2022)
- [Understanding the role of importance weighting for deep learning](https://mlsanity.com/paper/2103.15209/) (2021)
- [On the Optimality of Misspecified Spectral Algorithms](https://mlsanity.com/paper/2303.14942/) (2023)
- [A Short Survey on Importance Weighting for Machine Learning](https://mlsanity.com/paper/2403.10175/) (2024)
- [When is Importance Weighting Correction Needed for Covariate Shift Adaptation?](https://mlsanity.com/paper/2303.04020/) (2023)
- [Towards a Unified Analysis of Kernel-based Methods Under Covariate Shift](https://mlsanity.com/paper/2310.08237/) (2023)
- [On regularized Radon-Nikodym differentiation](https://mlsanity.com/paper/2308.07887/) (2023)
- [Estimating Unbounded Density Ratios: Applications in Error Control under Covariate Shift](https://mlsanity.com/paper/2504.01031/) (2025)
- [Generalization Analysis of Online Stochastic Gradient Descent for Overparameterized Two-Layer Neural Networks](https://mlsanity.com/paper/2407.07670/) (2024)
- [Spectral Algorithms under Covariate Shift](https://mlsanity.com/paper/2504.12625/) (2025)

## Built on by (most cited first, 3 of 3)

- [Towards regularized learning from functional data with covariate shift](https://mlsanity.com/paper/2601.21019/) (2026)
- [Convergence Analysis of Nystr"om Subsampling in Covariate Shift Adaptation for Misspecified case](https://mlsanity.com/paper/2606.22259/) (2026)
- [Learning under Target Shift: Optimal Density Ratio Estimation and Importance-Weighted Regression](https://mlsanity.com/paper/2609.15785/) (2026)

## Abstract (arXiv)

This paper investigates the convergence properties of spectral algorithms -- a class of regularization methods originating from inverse problems -- under covariate shift. In this setting, the marginal distributions of inputs differ between source and target domains, while the conditional distribution of outputs given inputs remains unchanged. To address this distributional mismatch, we incorporate importance weights, defined as the ratio of target to source densities, into the learning framework. This leads to a weighted spectral algorithm within a nonparametric regression setting in a reproducing kernel Hilbert space (RKHS). More importantly, in contrast to prior work that largely focuses on the well-specified setting, we provide a comprehensive theoretical analysis of the more challenging misspecified case, in which the target function does not belong to the RKHS. Under the assumption of uniformly bounded density ratios, we establish minimax-optimal convergence rates when the target function lies within the RKHS. For scenarios involving unbounded importance weights, we introduce a novel truncation technique that attains near-optimal convergence rates under mild regularity conditions, and we further extend these results to the misspecified regime. By addressing the intertwined challenges of covariate shift and model misspecification, this work extends classical kernel learning theory to more practical scenarios, providing a systematic framework for understanding their interaction.