mlsanityData to 7 Oct 2026

Mitigating Covariate Shift in Misspecified Regression with Applications to Reinforcement Learning

arXiv 2401.12216 ↗First posted 22 Jan 2024stat.ML cs.LG math.OCcited by 4

Not on the map. Why?

Abstract (arXiv)

A pervasive phenomenon in machine learning applications is distribution shift, where training and deployment conditions for a machine learning model differ. As distribution shift typically results in a degradation in performance, much attention has been devoted to algorithmic interventions that mitigate these detrimental effects. In this paper, we study the effect of distribution shift in the presence of model misspecification, specifically focusing on L∞-misspecified regression and adversarial covariate shift, where the regression target remains fixed while the covariate distribution changes arbitrarily. We show that empirical risk minimization, or standard least squares regression, can result in undesirable misspecification amplification where the error due to misspecification is amplified by the density ratio between the training and testing distributions. As our main result, we develop a new algorithm -- inspired by robust optimization techniques -- that avoids this undesirable behavior, resulting in no misspecification amplification while still obtaining optimal statistical rates. As applications, we use this regression procedure to obtain new guarantees in offline and online reinforcement learning with misspecification and establish new separations between previously studied structural conditions and notions of coverage.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 37 of its 66 references that are in the map. First posted on arXiv 22 Jan 2024.

Builds on

37 of 66 references.

References · 37

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  2. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  3. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
  4. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,232
  5. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 431
  6. 2018Fairness Without Demographics in Repeated Loss Minimizationcited by 363
  7. 2018Learning Models with Uniform Performance via Distributionally Robust Optimizationcited by 334
  8. 2020Performative Predictioncited by 318
  9. 2016Contextual Decision Processes with Low Bellman Rank are PAC-Learnablecited by 290
  10. 2019Information-Theoretic Considerations in Batch Reinforcement Learningcited by 285
  11. 2021Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalizationcited by 248
  12. 2021Bellman Eluder Dimension: New Rich Classes of RL Problems, and Sample-Efficient Algorithmscited by 222
25 more
  1. 2022Exploring Length Generalization in Large Language Modelscited by 204
  2. 2021Bilinear Classes: A Structural Framework for Provable Generalization in RLcited by 176
  3. 2013Non-stationary Stochastic Optimizationcited by 173
  4. 2019Is a Good Representation Sufficient for Sample Efficient Reinforcement Learning?cited by 140
  5. 2019Learning with Good Feature Representations in Bandits and in RL with a Generative Modelcited by 124
  6. 2022Unveiling Transformers with LEGO: a synthetic reasoning taskcited by 105
  7. 2020Batch Value-function Approximation with Only Realizabilitycited by 103
  8. 2018Practical Contextual Bandits with Regression Oraclescited by 92
  9. 2022The Role of Coverage in Online Reinforcement Learningcited by 82
  10. 2020Instance-Dependent Complexity of Contextual Bandits and Reinforcement Learning: A Disagreement-Based Perspectivecited by 78
  11. 2023Exposing Attention Glitches with Flip-Flop Language Modelingcited by 74
  12. 2021Offline Reinforcement Learning: Fundamental Barriers for Value Function Approximationcited by 69
  13. 2018Marginal Singularity, and the Benefits of Labels in Covariate-Shiftcited by 65
  14. 2022Optimally tackling covariate shift in RKHS-based nonparametric regressioncited by 53
  15. 2017Active Learning for Cost-Sensitive Classificationcited by 40
  16. 2021Near-Optimal Linear Regression under Distribution Shiftcited by 40
  17. 2015Learning with Square Loss: Localization through Offset Rademacher Complexitycited by 37
  18. 2022A new similarity measure for covariate shift with applications to nonparametric regressioncited by 33
  19. 2022Minimax Regret Optimization for Robust Machine Learning under Distribution Shiftcited by 31
  20. 2022First Steps Toward Understanding the Extrapolation of Nonlinear Models to Unseen Domainscited by 20
  21. 2019Comments on the Du-Kakade-Wang-Yang Lower Boundscited by 16
  22. 2023Maximum Likelihood Estimation is All You Need for Well-Specified Covariate Shiftcited by 14
  23. 2020Optimal oracle inequalities for solving projected fixed-point equationscited by 10
  24. 2023Toward L_∞-recovery of Nonlinear Functions: A Polynomial Sample Complexity Bound for Gaussian Random Fieldscited by 4
  25. 2023The Optimal Approximation Factors in Misspecified Off-Policy Value Function Estimationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 1.

4 papers.

  1. 2025Model Selection for Off-policy Evaluation: New Algorithms and Experimental Protocolcited by 6
  2. 2025Representative, Informative, and De-Amplifying: Requirements for Robust Bayesian Active Learning under Model Misspecificationcited by 1
  3. 2026Online KL-Regularized Reinforcement Learning with Function Approximation under Misspecificationcited by 1
  4. 2024Misspecified Q-Learning with Sparse Linear Function Approximation: Tight Bounds on Approximation Errorcited by 0