mlsanityData to 7 Oct 2026

Marginal Singularity, and the Benefits of Labels in Covariate-Shift

arXiv 1803.01833 ↗First posted 5 Mar 2018stat.ML cs.LGcited by 65

Not on the map. Why?

Abstract (arXiv)

We present new minimax results that concisely capture the relative benefits of source and target labeled data, under covariate-shift. Namely, we show that the benefits of target labels are controlled by a transfer-exponent γ that encodes how singular Q is locally w.r.t. P, and interestingly allows situations where transfer did not seem possible under previous insights. In fact, our new minimax analysis - in terms of γ - reveals a continuum of regimes ranging from situations where target labels have little benefit, to regimes where target labels dramatically improve classification. We then show that a recently proposed semi-supervised procedure can be extended to adapt to unknown γ, and therefore requests labels only when beneficial, while achieving minimax transfer rates.

Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 16; 2023: 14; 2024: 16; 2025: 11; 2026: 4.

65 papers.

Sort:

By area:

  1. 2022Optimally tackling covariate shift in RKHS-based nonparametric regressioncited by 53
  2. 2023AdaNPC: Exploring Non-Parametric Classifier for Test-Time Adaptationcited by 48
  3. 2021How Fine-Tuning Allows for Effective Meta-Learningcited by 41
  4. 2025CARROT: A Cost Aware Rate Optimal Routercited by 34
  5. 2022A new similarity measure for covariate shift with applications to nonparametric regressioncited by 33
  6. 2022Adaptive and Robust Multi-Task Learningcited by 33
  7. 2021Simple and near-optimal algorithms for hidden stratification and multi-group learningcited by 26
  8. 2024Understanding Forgetting in Continual Learning with Linear Regressioncited by 26
  9. 2022The Power and Limitation of Pretraining-Finetuning for Linear Regression under Covariate Shiftcited by 25
  10. 2022Transfer Learning for Contextual Multi-armed Banditscited by 24
  11. 2023Pseudo-Labeling for Kernel Ridge Regression under Covariate Shiftcited by 17
  12. 2022Domain-Specific Risk Minimization for Out-of-Distribution Generalizationcited by 15
38 more
  1. 2023Adapting to Continuous Covariate Shift via Online Density Ratio Estimationcited by 15
  2. 2024A transfer learning framework for weak-to-strong generalizationcited by 14
  3. 2023Towards a Unified Analysis of Kernel-based Methods Under Covariate Shiftcited by 12
  4. 2023When is Importance Weighting Correction Needed for Covariate Shift Adaptation?cited by 11
  5. 2025Task Generalization With AutoRegressive Compositional Structure: Can Learning From D Tasks Generalize to DT Tasks?cited by 11
  6. 2022On the Limitations of General Purpose Domain Generalisation Methodscited by 10
  7. 2022A Class of Geometric Structures in Transfer Learning: Minimax Bounds and Optimalitycited by 9
  8. 2023Limits of Model Selection under Transfer Learningcited by 9
  9. 2023Model-free Test Time Adaptation for Out-Of-Distribution Detectioncited by 9
  10. 2024Efficient Discrepancy Testing for Learning with Distribution Shiftcited by 9
  11. 2022Breaking Correlation Shift via Conditional Invariant Regularizercited by 7
  12. 2023Best-Effort Adaptationcited by 7
  13. 2024Transfer Learning Beyond Bounded Density Ratioscited by 6
  14. 2021Universal and data-adaptive algorithms for model selection in linear contextual banditscited by 5
  15. 2023Nearest Neighbor Sampling for Covariate Shift Adaptationcited by 5
  16. 2021Adaptive transfer learningcited by 4
  17. 2022On Causality in Domain Adaptation and Semi-Supervised Learning: an Information-Theoretic Analysis for Parametric Modelscited by 4
  18. 2024Mitigating Covariate Shift in Misspecified Regression with Applications to Reinforcement Learningcited by 4
  19. 2024Causally Inspired Regularization Enables Domain General Representationscited by 4
  20. 2024Fast Computation of Leave-One-Out Cross-Validation for k-NN Regressioncited by 4
  21. 2024High-Dimensional Kernel Methods under Covariate Shift: Data-Dependent Implicit Regularizationcited by 4
  22. 2024Minimax And Adaptive Transfer Learning for Nonparametric Classification under Distributed Differential Privacy Constraintscited by 4
  23. 2024Deep Transfer Learning: Model Framework and Error Analysiscited by 4
  24. 2025Computational Efficiency under Covariate Shift in Kernel Ridge Regressioncited by 4
  25. 2022Undersampling is a Minimax Optimal Robustness Intervention in Nonparametric Classificationcited by 3
  26. 2022Target Conditioned Representation Independence (TCRI); From Domain-Invariant to Domain-General Representationscited by 3
  27. 2023Classification Tree Pruning Under Covariate Shiftcited by 3
  28. 2023Double Debiased Covariate Shift Adaptation Robust to Density-Ratio Estimationcited by 3
  29. 2023Invariance assumptions for class distribution estimationcited by 3
  30. 2024On the design-dependent suboptimality of the Lassocited by 3
  31. 2024Sharp analysis of out-of-distribution error for "importance-weighted" estimators in the overparameterized regimecited by 3
  32. 2024Help Me Help You: The Aggregate Value of Source and Target Data in Transfer Learningcited by 3
  33. 2025Conformal Prediction Under Generalized Covariate Shift with Posterior Driftcited by 3
  34. 2022Connecting sufficient conditions for domain adaptation: source-guided uncertainty, relaxed divergences and discrepancy localizationcited by 2
  35. 2025Fixed-Gaussian Spectral Algorithms: Minimax Optimal Rates for Misspecified Learning and Transfercited by 2
  36. 2025Neyman-Pearson Classification under Both Null and Alternative Distributions Shiftcited by 2
  37. 2022Automatically Score Tissue Images Like a Pathologist by Transfer Learningcited by 1
  38. 2022Blessings and Curses of Covariate Shifts: Adversarial Learning Dynamics, Directional Convergence, and Equilibriacited by 1

50 most cited of 65 All 65 (JSON)