# Towards a Unified Analysis of Kernel-based Methods Under Covariate Shift

arXiv: https://arxiv.org/abs/2310.08237 · HTML: https://mlsanity.com/paper/2310.08237/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 56 references that are in the map.
- First posted on arXiv 12 Oct 2023.

## Builds on (most cited first, 12 of 12)

- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [What is the Effect of Importance Weighting in Deep Learning?](https://mlsanity.com/paper/1812.03372/) (2018)
- [Marginal Singularity, and the Benefits of Labels in Covariate-Shift](https://mlsanity.com/paper/1803.01833/) (2018)
- [Optimally tackling covariate shift in RKHS-based nonparametric regression](https://mlsanity.com/paper/2205.02986/) (2022)
- [Non-Negative Bregman Divergence Minimization for Deep Direct Density Ratio Estimation](https://mlsanity.com/paper/2006.06979/) (2020)
- [Near-Optimal Linear Regression under Distribution Shift](https://mlsanity.com/paper/2106.12108/) (2021)
- [A new similarity measure for covariate shift with applications to nonparametric regression](https://mlsanity.com/paper/2202.02837/) (2022)
- [Randomized sketches for kernels: Fast and optimal non-parametric regression](https://mlsanity.com/paper/1501.06195/) (2015)
- [When is Importance Weighting Correction Needed for Covariate Shift Adaptation?](https://mlsanity.com/paper/2303.04020/) (2023)
- [Bayesian multi-domain learning for cancer subtype discovery from next-generation sequencing count data](https://mlsanity.com/paper/1810.09433/) (2018)

## Built on by (most cited first, 13 of 13)

- [The Expressibility of Polynomial based Attention Scheme](https://mlsanity.com/paper/2310.20051/) (2023)
- [High-Dimensional Kernel Methods under Covariate Shift: Data-Dependent Implicit Regularization](https://mlsanity.com/paper/2406.03171/) (2024)
- [Optimal Kernel Quantile Learning with Random Features](https://mlsanity.com/paper/2408.13591/) (2024)
- [Transfer Learning for Kernel-based Regression](https://mlsanity.com/paper/2310.13966/) (2023)
- [Sharp analysis of out-of-distribution error for "importance-weighted" estimators in the overparameterized regime](https://mlsanity.com/paper/2405.06546/) (2024)
- [Demystifying Spectral Bias on Real-World Data](https://mlsanity.com/paper/2406.02663/) (2024)
- [Spectral Algorithms under Covariate Shift](https://mlsanity.com/paper/2504.12625/) (2025)
- [Spectral Algorithms in Misspecified Regression: Convergence under Covariate Shift](https://mlsanity.com/paper/2509.05106/) (2025)
- [Harnessing the Power of Vicinity-Informed Analysis for Classification under Covariate Shift](https://mlsanity.com/paper/2405.16906/) (2024)
- [Minimax Optimal Two-Stage Algorithm For Moment Estimation Under Covariate Shift](https://mlsanity.com/paper/2506.23453/) (2025)
- [Provable Target Sample Complexity Improvements as Pre-Trained Models Scale](https://mlsanity.com/paper/2602.04233/) (2026)
- [Adaptive deep nonparametric regression from dependent data under covariate shift](https://mlsanity.com/paper/2607.20309/) (2026)
- [Latent-Space Importance Weighting under Covariate Shift](https://mlsanity.com/paper/2610.15571/) (2026)

## Abstract (arXiv)

Covariate shift occurs prevalently in practice, where the input distributions of the source and target data are substantially different. Despite its practical importance in various learning problems, most of the existing methods only focus on some specific learning tasks and are not well validated theoretically and numerically. To tackle this problem, we propose a unified analysis of general nonparametric methods in a reproducing kernel Hilbert space (RKHS) under covariate shift. Our theoretical results are established for a general loss belonging to a rich loss function family, which includes many commonly used methods as special cases, such as mean regression, quantile regression, likelihood-based classification, and margin-based classification. Two types of covariate shift problems are the focus of this paper and the sharp convergence rates are established for a general loss function to provide a unified theoretical analysis, which concurs with the optimal results in literature where the squared loss is used. Extensive numerical studies on synthetic and real examples confirm our theoretical findings and further illustrate the effectiveness of our proposed method.