mlsanityData to 14 Oct 2026

Near-Optimal Linear Regression under Distribution Shift

arXiv 2106.12108 ↗First posted 23 Jun 2021cs.LG stat.MLcited by 42

Not on the map. Why?

Abstract (arXiv)

Transfer learning is essential when sufficient data comes from the source domain, with scarce labeled data from the target domain. We develop estimators that achieve minimax linear risk for linear regression problems under distribution shift. Our algorithms cover different transfer learning settings including covariate shift and model shift. We also consider when data are generated from either linear or general nonlinear models. We show that linear minimax estimators are within an absolute constant of the minimax risk even among nonlinear estimators for various source/target distributions.

Key facts. Cited by 42 papers in the map (data through 14 Oct 2026). Builds on 25 of its 70 references that are in the map. First posted on arXiv 23 Jun 2021.

Builds on

25 of 70 references.

References · 25

  1. 2015Domain-Adversarial Training of Neural Networkscited by 3,044
  2. 2019Invariant Risk Minimizationcited by 1,854
  3. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,247
  4. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,200
  5. 2016Deep Transfer Learning with Joint Adaptation Networkscited by 612
  6. 2018Detecting and Correcting for Label Shift with Black Box Predictorscited by 366
  7. 2020The Risks of Invariant Risk Minimizationcited by 262
  8. 2020Few-Shot Learning via Learning the Representation, Provablycited by 214
  9. 2018Do CIFAR-10 Classifiers Generalize to CIFAR-10?cited by 200
  10. 2020Understanding Self-Training for Gradual Domain Adaptationcited by 149
  11. 2020Domain Adaptation with Conditional Distribution Matching and Generalized Label Shiftcited by 125
  12. 2021Does Invariant Risk Minimization Capture Invariance?cited by 116
13 more
  1. 2019Generalizing to unseen domains via distribution matchingcited by 110
  2. 2016A Geometric Analysis of Phase Retrievalcited by 92
  3. 2020On the Value of Target Data in Transfer Learningcited by 65
  4. 2020Robustness to Spurious Correlations via Human Annotationscited by 62
  5. 2020Empirical or Invariant Risk Minimization? A Sample Complexity Perspectivecited by 62
  6. 2019Domain Adaptation with Asymmetrically-Relaxed Distribution Alignmentcited by 61
  7. 2015A Century of Portraits: A Visual Historical Record of American High School Yearbookscited by 54
  8. 2021A Theory of Label Propagation for Subpopulation Shiftcited by 48
  9. 2019On Learning Invariant Representation for Domain Adaptationcited by 39
  10. 2020Minimax Lower Bounds for Transfer Learning with Linear and One-hidden Layer Neural Networkscited by 30
  11. 2021Iterative Feature Matching: Toward Provable Domain Generalization with Logarithmic Environmentscited by 30
  12. 2016Generative Adversarial Nets from a Density Ratio Estimation Perspectivecited by 24
  13. 2021Gradual Domain Adaptation in the Wild:When Intermediate Distributions are Absentcited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 7; 2023: 14; 2024: 11; 2025: 7; 2026: 3.

42 papers.

Sort:

By area:

  1. 2022Virtual Homogeneity Learning: Defending against Data Heterogeneity in Federated Learningcited by 52
  2. 2022Adaptive and Robust Multi-Task Learningcited by 34
  3. 2022Robust Fine-Tuning of Deep Neural Networks with Hessian-based Generalization Guaranteescited by 34
  4. 2022A new similarity measure for covariate shift with applications to nonparametric regressioncited by 33
  5. 2024A Statistical Theory of Regularization-Based Continual Learningcited by 33
  6. 2022The Power and Limitation of Pretraining-Finetuning for Linear Regression under Covariate Shiftcited by 26
  7. 2023Fixed Design Analysis of Regularization-Based Continual Learningcited by 24
  8. 2022First Steps Toward Understanding the Extrapolation of Nonlinear Models to Unseen Domainscited by 21
  9. 2023Pseudo-Labeling for Kernel Ridge Regression under Covariate Shiftcited by 17
  10. 2023Adapting to Continuous Covariate Shift via Online Density Ratio Estimationcited by 15
  11. 2023Maximum Likelihood Estimation is All You Need for Well-Specified Covariate Shiftcited by 15
  12. 2023When is Importance Weighting Correction Needed for Covariate Shift Adaptation?cited by 13
30 more
  1. 2023Towards a Unified Analysis of Kernel-based Methods Under Covariate Shiftcited by 13
  2. 2024Generalizing across Temporal Domains with Koopman Operatorscited by 13
  3. 2023Demystifying Disagreement-on-the-Line in High Dimensionscited by 12
  4. 2024TransFusion: Covariate-Shift Robust Transfer Learning for High-Dimensional Regressioncited by 11
  5. 2025Task Generalization With AutoRegressive Compositional Structure: Can Learning From D Tasks Generalize to DT Tasks?cited by 11
  6. 2024Controllable Prompt Tuning For Balancing Group Distributional Robustnesscited by 10
  7. 2023Moderately Distributional Exploration for Domain Generalizationcited by 8
  8. 2023Double Debiased Covariate Shift Adaptation Robust to Density-Ratio Estimationcited by 6
  9. 2023Strategic Distribution Shift of Interacting Agents via Coupled Gradient Flowscited by 5
  10. 2024Sketchy Moment Matching: Toward Fast and Provable Data Selection for Finetuningcited by 5
  11. 2025Memory-Statistics Tradeoff in Continual Learning with Structural Regularizationcited by 5
  12. 2024Mitigating Covariate Shift in Misspecified Regression with Applications to Reinforcement Learningcited by 4
  13. 2025Learning Neural Networks with Distribution Shift: Efficiently Certifiable Guaranteescited by 4
  14. 2025Computational Efficiency under Covariate Shift in Kernel Ridge Regressioncited by 4
  15. 2023Tight Rates in Supervised Outlier Transfer Learningcited by 3
  16. 2024On the design-dependent suboptimality of the Lassocited by 3
  17. 2025Transfer Learning for Benign Overfitting in High-Dimensional Linear Regressioncited by 3
  18. 2023Statistical Learning under Heterogeneous Distribution Shiftcited by 2
  19. 2023Out of the Ordinary: Spectrally Adapting Regression for Covariate Shiftcited by 2
  20. 2024Bridging Domains with Approximately Shared Featurescited by 2
  21. 2025Fixed-Gaussian Spectral Algorithms: Minimax Optimal Rates for Misspecified Learning and Transfercited by 2
  22. 2024Domain Adaptation Using Pseudo Labelscited by 1
  23. 2025Knowledge Distillation of Uncertainty using Deep Latent Factor Modelcited by 1
  24. 2022Beyond Ridge Regression for Distribution-Free Datacited by 0
  25. 2023On Prediction Feature Assignment in the Heckman Selection Modelcited by 0
  26. 2024Learning When the Concept Shifts: Confounding, Invariance, and Dimension Reductioncited by 0
  27. 2024Prediction Accuracy & Reliability: Classification and Object Localization under Distribution Shiftcited by 0
  28. 2026A nonparametric two-sample test using a parametric integral probability metriccited by 0
  29. 2026Why Linear Models Can Still Work in a Complex and Changing Worldcited by 0
  30. 2026Feature Normalization under Finite Training Budgetscited by 0